-----============= acceptance-small: conf-sanity ============----- Mon Sep 14 06:38:51 EDT 2026
MGS_OS_ID_LIKE=rhel centos fedora rocky
MGS_OS_VERSION_ID=8.10
MGS_OS_ID=rocky
MGS_OS_VERSION_CODE=134873088
MDS1_OS_VERSION_ID=8.10
MDS1_OS_VERSION_CODE=134873088
MDS1_OS_ID_LIKE=rhel centos fedora rocky
MDS1_OS_ID=rocky
OST1_OS_VERSION_CODE=134873088
OST1_OS_ID_LIKE=rhel centos fedora rocky
OST1_OS_VERSION_ID=8.10
OST1_OS_ID=rocky
CLIENT_OS_ID=rocky
CLIENT_OS_VERSION_CODE=134873088
CLIENT_OS_VERSION_ID=8.10
CLIENT_OS_ID_LIKE=rhel centos fedora rocky
oleg135-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex
- see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683432 >= 34553653) for LU-16938, go 81
- see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683432 >= 34554143) for LU-18348, go 32b 32c
- see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683432 >= 34617107) for LU-17920, go 123ae 123ag
- see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683432 >= 34681950) for LU-19740, go 115
excepting tests: 32newtarball
skipping tests SLOW=no: 45 69 106 111 114
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:-f)
Stopping client oleg135-client.virtnet /mnt/lustre opts:-f
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:-f)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg135-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg135-server
oleg135-server: oleg135-server.virtnet: executing set_hostid
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Formatting mgs, mds, osts
Format mds1: /dev/mapper/mds1_flakey
Format ost1: /dev/mapper/ost1_flakey
Format ost2: /dev/mapper/ost2_flakey
start mds service on oleg135-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Commit the device label on /dev/mapper/mds1_flakey
Started lustre-MDT0000
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg135-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Commit the device label on /dev/mapper/ost1_flakey
Started lustre-OST0000
oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
oleg135-server: oleg135-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50
oleg135-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec
stop ost1 service on oleg135-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg135-server
stop mds service on oleg135-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server

== conf-sanity test 157a: test allow_register for MDT registration and MGS restart ========================================================== 06:41:47 (1789382507)
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:)
start mds service on oleg135-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg135-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-OST0000
oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg135-client.virtnet:  -o user_xattr,flock 192.168.201.135@tcp:/lustre /mnt/lustre
allow_register=unlimited
allow_register=0
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-mdt1_2
Target:     fs184:MDT0001
Index:      1
Lustre FS:  fs184
Mount type: ldiskfs
Flags:      0x61
              (MDT first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: mgsnode=192.168.201.135@tcp:192.168.201.135@tcp sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs184:MDT0001
	kilobytes     200000
	options       -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs184:MDT0001 -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg135-server: mount.lustre: mount /dev/mapper/fs2mds_flakey at /mnt/lustre-fs2mds failed: Permission denied
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 13
Start of /dev/mapper/fs2mds_flakey on fs2mds failed 13
allow_register=unlimited
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-mdt1_2
Started fs184-MDT0001
mdt.fs184-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg135-server
allow_register=0
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started fs184-MDT0001
mdt.fs184-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg135-server
stop mds service on oleg135-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
start mds service on oleg135-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
affected facets: mgs
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started fs184-MDT0001
mdt.fs184-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg135-server
allow_register functionality test completed successfully
Cleaning up test 157a...
umount lustre on /mnt/lustre.....
Stopping client oleg135-client.virtnet /mnt/lustre (opts:)
oleg135-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg135-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg135-server
stop mds service on oleg135-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
PASS 157a (261s)

== conf-sanity test 157b: verify allow_register (block new OSTs, allow existing) ========================================================== 06:46:08 (1789382768)
start mds service on oleg135-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg135-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg135-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-OST0000
oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg135-client.virtnet:  -o user_xattr,flock 192.168.201.135@tcp:/lustre /mnt/lustre
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs24256:OST0000
Index:      0
Lustre FS:  fs24256
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.135@tcp:192.168.201.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs24256:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs24256:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs24256-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs24256-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
obdfilter.fs24256-OST0000.recovery_status=status: INACTIVE
allow_register=0
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg135-server
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs24256-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started fs24256-OST0000
obdfilter.fs24256-OST0000.recovery_status=status: INACTIVE
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost2_2
Target:     fs24256:OST0001
Index:      1
Lustre FS:  fs24256
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.135@tcp:192.168.201.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs24256:OST0001
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs24256:OST0001 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
oleg135-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 13
oleg135-server: error: set_param: param_path 'seq/cli-fs24256:OST0001-super/width': No such file or directory
oleg135-server: error: set_param: setting 'seq/cli-fs24256:OST0001-super/width'='65536': No such file or directory
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13
allow_register=unlimited
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs24256-OST0001-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost2_2
Started fs24256-OST0001
/mnt/lustre-fs3ost: 83.1 MiB (87146496 bytes) trimmed
obdfilter.fs24256-OST0000.recovery_status=status: INACTIVE
obdfilter.fs24256-OST0001.recovery_status=status: INACTIVE
allow_register=0
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg135-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg135-server
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs24256-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started fs24256-OST0000
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs24256-OST0001-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started fs24256-OST0001
Passed: Existing targets can re-register, new targets blocked
Cleaning up test 157b...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg135-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg135-server
umount lustre on /mnt/lustre.....
Stopping client oleg135-client.virtnet /mnt/lustre (opts:)
oleg135-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg135-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg135-server
stop mds service on oleg135-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
PASS 157b (333s)

== conf-sanity test 157c: test allow_register timeout functionality ========================================================== 06:51:41 (1789383101)
start mds service on oleg135-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg135-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg135-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-OST0000
oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg135-client.virtnet:  -o user_xattr,flock 192.168.201.135@tcp:/lustre /mnt/lustre
allow_register=unlimited
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs32324:OST0000
Index:      0
Lustre FS:  fs32324
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.135@tcp:192.168.201.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs32324:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs32324:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs32324-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs32324-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
allow_register=60s
obdfilter.fs32324-OST0000.recovery_status=status: INACTIVE
Waiting for allow_register timeout to expire
Waiting 66s for '1'
Waiting 56s for '1'
Waiting 46s for '1'
Waiting 36s for '1'
Waiting 26s for '1'
Waiting 16s for '1'
Updated after 60s: want '1' got '1'
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost2_2
Target:     fs32324:OST0001
Index:      1
Lustre FS:  fs32324
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.135@tcp:192.168.201.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs32324:OST0001
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs32324:OST0001 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
oleg135-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 13
oleg135-server: error: set_param: param_path 'seq/cli-fs32324:OST0001-super/width': No such file or directory
oleg135-server: error: set_param: setting 'seq/cli-fs32324:OST0001-super/width'='65536': No such file or directory
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13
allow_register=unlimited
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs32324-OST0001-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost2_2
Started fs32324-OST0001
/mnt/lustre-fs3ost: 83.1 MiB (87146496 bytes) trimmed
obdfilter.fs32324-OST0000.recovery_status=status: INACTIVE
obdfilter.fs32324-OST0001.recovery_status=status: INACTIVE
Cleaning up test 157c...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg135-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg135-server
umount lustre on /mnt/lustre.....
Stopping client oleg135-client.virtnet /mnt/lustre (opts:)
oleg135-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg135-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg135-server
stop mds service on oleg135-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
PASS 157c (303s)

== conf-sanity test 157d: test allow_register -P persistence and legacy 1->unlimited ========================================================== 06:56:44 (1789383404)
start mds service on oleg135-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg135-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg135-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-OST0000
oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg135-client.virtnet:  -o user_xattr,flock 192.168.201.135@tcp:/lustre /mnt/lustre
Waiting 90s for '0'
Updated after 8s: want '0' got '0'
oleg135-server: error: executing set_param: Invalid argument
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 22
oleg135-server: error: executing set_param: Invalid argument
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 22
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:)
Stopping client oleg135-client.virtnet /mnt/lustre opts:
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg135-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
affected facets: mgs
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:)
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg135-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
affected facets: mgs
allow_register persistence and legacy 1->unlimited OK
Cleaning up test 157d...
umount lustre on /mnt/lustre.....
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg135-server
stop mds service on oleg135-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
PASS 157d (341s)

== conf-sanity test 157e: test allow_register mgs module parameter boot-time enforcement ========================================================== 07:02:25 (1789383745)
start mds service on oleg135-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg135-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-OST0000
oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg135-client.virtnet:  -o user_xattr,flock 192.168.201.135@tcp:/lustre /mnt/lustre
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:)
Stopping client oleg135-client.virtnet /mnt/lustre opts:
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg135-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
allow_register=60s
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg135-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg135-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
mgs allow_register module parameter OK
Cleaning up test 157e...
oleg135-server: error: executing set_param: No such file or directory
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
umount lustre on /mnt/lustre.....
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg135-server
stop mds service on oleg135-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
PASS 157e (385s)

== conf-sanity test 157f: test allow_register auto-arm default 24-hour window ========================================================== 07:08:50 (1789384130)
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:)
LNET unconfigure error 22: Invalid argument
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
oleg135-server: LNET unconfigure error 22: Invalid argument
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg135-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
oleg135-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg135-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Started lustre-OST0000
oleg135-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg135-client.virtnet:  -o user_xattr,flock 192.168.201.135@tcp:/lustre /mnt/lustre
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-mdt1_2
Target:     fs17905:MDT0001
Index:      1
Lustre FS:  fs17905
Mount type: ldiskfs
Flags:      0x61
              (MDT first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: mgsnode=192.168.201.135@tcp:192.168.201.135@tcp sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs17905:MDT0001
	kilobytes     200000
	options       -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs17905:MDT0001 -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-mdt1_2
Started fs17905-MDT0001
mdt.fs17905-MDT0001.recovery_status=status: INACTIVE
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs25574:OST0000
Index:      0
Lustre FS:  fs25574
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.135@tcp:192.168.201.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs25574:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs25574:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs25574-OST0000-super.width=65536
oleg135-server: oleg135-server.virtnet: executing set_default_debug -1 all
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs25574-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
obdfilter.fs25574-OST0000.recovery_status=status: INACTIVE
allow_register auto-arm 24h window test completed successfully
Cleaning up test 157f...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg135-server
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg135-server
umount lustre on /mnt/lustre.....
Stopping client oleg135-client.virtnet /mnt/lustre (opts:)
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg135-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg135-server
stop mds service on oleg135-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg135-server
unloading modules via unload_modules_local on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing unload_modules_local
oleg135-server: kmemleak disabled
modules unloaded.
PASS 157f (262s)
Stopping clients: oleg135-client.virtnet /mnt/lustre (opts:-f)
Stopping clients: oleg135-client.virtnet /mnt/lustre2 (opts:-f)
pdsh@oleg135-client: oleg135-server: ssh exited with exit code 2
oleg135-server: oleg135-server.virtnet: executing set_hostid
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg135-server'
oleg135-server: oleg135-server.virtnet: executing load_modules_local
oleg135-server: kmemleak disabled
oleg135-server: Loading modules from /home/green/git/lustre-release/lustre
oleg135-server: detected 4 online CPUs by sysfs
oleg135-server: MODOPTS_LIBCFS=
oleg135-server: Force libcfs to create 2 CPU partitions
oleg135-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg135-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg135-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg135-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Formatting mgs, mds, osts
Format mds1: /dev/mapper/mds1_flakey
Format ost1: /dev/mapper/ost1_flakey
Format ost2: /dev/mapper/ost2_flakey
== conf-sanity test complete, duration 2127 sec ========== 07:14:20 (1789384460)
=== conf-sanity: start cleanup 07:14:23 (1789384463) ===
=== conf-sanity: finish cleanup 07:14:27 (1789384467) ===