-----============= acceptance-small: conf-sanity ============----- Mon Sep 14 06:12:10 EDT 2026
MGS_OS_ID_LIKE=rhel centos fedora rocky
MGS_OS_VERSION_ID=8.10
MGS_OS_ID=rocky
MGS_OS_VERSION_CODE=134873088
MDS1_OS_VERSION_ID=8.10
MDS1_OS_VERSION_CODE=134873088
MDS1_OS_ID_LIKE=rhel centos fedora rocky
MDS1_OS_ID=rocky
OST1_OS_VERSION_CODE=134873088
OST1_OS_ID_LIKE=rhel centos fedora rocky
OST1_OS_VERSION_ID=8.10
OST1_OS_ID=rocky
CLIENT_OS_ID=rocky
CLIENT_OS_VERSION_CODE=134873088
CLIENT_OS_VERSION_ID=8.10
CLIENT_OS_ID_LIKE=rhel centos fedora rocky
oleg120-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex
- see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683432 >= 34553653) for LU-16938, go 81
- see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683432 >= 34554143) for LU-18348, go 32b 32c
- see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683432 >= 34617107) for LU-17920, go 123ae 123ag
- see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683432 >= 34681950) for LU-19740, go 115
excepting tests: 32newtarball
skipping tests SLOW=no: 45 69 106 111 114
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:-f)
Stopping client oleg120-client.virtnet /mnt/lustre opts:-f
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:-f)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg120-server
oleg120-server: oleg120-server.virtnet: executing set_hostid
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Formatting mgs, mds, osts
Format mds1: /dev/mapper/mds1_flakey
Format ost1: /dev/mapper/ost1_flakey
Format ost2: /dev/mapper/ost2_flakey
start mds service on oleg120-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Commit the device label on /dev/mapper/mds1_flakey
Started lustre-MDT0000
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
oleg120-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg120-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Commit the device label on /dev/mapper/ost1_flakey
Started lustre-OST0000
oleg120-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
oleg120-server: oleg120-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50
oleg120-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec
stop ost1 service on oleg120-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server
stop mds service on oleg120-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server

== conf-sanity test 157a: test allow_register for MDT registration and MGS restart ========================================================== 06:15:23 (1789380923)
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:)
start mds service on oleg120-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
oleg120-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg120-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-OST0000
oleg120-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg120-client.virtnet:  -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre
allow_register=unlimited
allow_register=0
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-mdt1_2
Target:     fs8608:MDT0001
Index:      1
Lustre FS:  fs8608
Mount type: ldiskfs
Flags:      0x61
              (MDT first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: mgsnode=192.168.201.120@tcp:192.168.201.120@tcp sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs8608:MDT0001
	kilobytes     200000
	options       -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs8608:MDT0001 -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg120-server: mount.lustre: mount /dev/mapper/fs2mds_flakey at /mnt/lustre-fs2mds failed: Permission denied
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 13
Start of /dev/mapper/fs2mds_flakey on fs2mds failed 13
allow_register=unlimited
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-mdt1_2
Started fs8608-MDT0001
mdt.fs8608-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg120-server
allow_register=0
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started fs8608-MDT0001
mdt.fs8608-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg120-server
stop mds service on oleg120-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
start mds service on oleg120-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
affected facets: mgs
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started fs8608-MDT0001
mdt.fs8608-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg120-server
allow_register functionality test completed successfully
Cleaning up test 157a...
umount lustre on /mnt/lustre.....
Stopping client oleg120-client.virtnet /mnt/lustre (opts:)
oleg120-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg120-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server
stop mds service on oleg120-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
PASS 157a (306s)

== conf-sanity test 157b: verify allow_register (block new OSTs, allow existing) ========================================================== 06:20:29 (1789381229)
start mds service on oleg120-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg120-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
oleg120-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg120-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-OST0000
oleg120-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg120-client.virtnet:  -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs32605:OST0000
Index:      0
Lustre FS:  fs32605
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.120@tcp:192.168.201.120@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs32605:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs32605:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs32605-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs32605-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
obdfilter.fs32605-OST0000.recovery_status=status: INACTIVE
allow_register=0
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg120-server
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs32605-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started fs32605-OST0000
obdfilter.fs32605-OST0000.recovery_status=status: INACTIVE
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost2_2
Target:     fs32605:OST0001
Index:      1
Lustre FS:  fs32605
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.120@tcp:192.168.201.120@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs32605:OST0001
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs32605:OST0001 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
oleg120-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 13
oleg120-server: error: set_param: param_path 'seq/cli-fs32605:OST0001-super/width': No such file or directory
oleg120-server: error: set_param: setting 'seq/cli-fs32605:OST0001-super/width'='65536': No such file or directory
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13
allow_register=unlimited
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs32605-OST0001-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost2_2
Started fs32605-OST0001
/mnt/lustre-fs3ost: 83.1 MiB (87146496 bytes) trimmed
obdfilter.fs32605-OST0000.recovery_status=status: INACTIVE
obdfilter.fs32605-OST0001.recovery_status=status: INACTIVE
allow_register=0
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg120-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg120-server
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs32605-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started fs32605-OST0000
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs32605-OST0001-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started fs32605-OST0001
Passed: Existing targets can re-register, new targets blocked
Cleaning up test 157b...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg120-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg120-server
umount lustre on /mnt/lustre.....
Stopping client oleg120-client.virtnet /mnt/lustre (opts:)
oleg120-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg120-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server
stop mds service on oleg120-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
PASS 157b (372s)

== conf-sanity test 157c: test allow_register timeout functionality ========================================================== 06:26:41 (1789381601)
start mds service on oleg120-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg120-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
oleg120-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg120-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-OST0000
oleg120-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg120-client.virtnet:  -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre
allow_register=unlimited
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs5192:OST0000
Index:      0
Lustre FS:  fs5192
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.120@tcp:192.168.201.120@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs5192:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs5192:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs5192-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs5192-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
allow_register=60s
obdfilter.fs5192-OST0000.recovery_status=status: INACTIVE
Waiting for allow_register timeout to expire
Waiting 65s for '1'
Waiting 55s for '1'
Waiting 45s for '1'
Updated after 55s: want '1' got '1'
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost2_2
Target:     fs5192:OST0001
Index:      1
Lustre FS:  fs5192
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.120@tcp:192.168.201.120@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs5192:OST0001
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs5192:OST0001 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
oleg120-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 13
oleg120-server: error: set_param: param_path 'seq/cli-fs5192:OST0001-super/width': No such file or directory
oleg120-server: error: set_param: setting 'seq/cli-fs5192:OST0001-super/width'='65536': No such file or directory
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13
allow_register=unlimited
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs5192-OST0001-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost2_2
Started fs5192-OST0001
/mnt/lustre-fs3ost: 83.1 MiB (87146496 bytes) trimmed
obdfilter.fs5192-OST0000.recovery_status=status: INACTIVE
obdfilter.fs5192-OST0001.recovery_status=status: INACTIVE
Cleaning up test 157c...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg120-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg120-server
umount lustre on /mnt/lustre.....
Stopping client oleg120-client.virtnet /mnt/lustre (opts:)
oleg120-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg120-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server
stop mds service on oleg120-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
PASS 157c (326s)

== conf-sanity test 157d: test allow_register -P persistence and legacy 1->unlimited ========================================================== 06:32:07 (1789381927)
start mds service on oleg120-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg120-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
oleg120-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg120-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-OST0000
oleg120-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg120-client.virtnet:  -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre
Waiting 90s for '0'
Updated after 9s: want '0' got '0'
oleg120-server: error: executing set_param: Invalid argument
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22
oleg120-server: error: executing set_param: Invalid argument
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:)
Stopping client oleg120-client.virtnet /mnt/lustre opts:
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg120-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
affected facets: mgs
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:)
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg120-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
affected facets: mgs
allow_register persistence and legacy 1->unlimited OK
Cleaning up test 157d...
umount lustre on /mnt/lustre.....
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg120-server
stop mds service on oleg120-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
PASS 157d (334s)

== conf-sanity test 157e: test allow_register mgs module parameter boot-time enforcement ========================================================== 06:37:41 (1789382261)
start mds service on oleg120-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
oleg120-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg120-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-OST0000
oleg120-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg120-client.virtnet:  -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:)
Stopping client oleg120-client.virtnet /mnt/lustre opts:
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg120-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
allow_register=60s
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg120-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg120-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
mgs allow_register module parameter OK
Cleaning up test 157e...
oleg120-server: error: executing set_param: No such file or directory
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
umount lustre on /mnt/lustre.....
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg120-server
stop mds service on oleg120-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
PASS 157e (368s)

== conf-sanity test 157f: test allow_register auto-arm default 24-hour window ========================================================== 06:43:50 (1789382630)
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:)
LNET unconfigure error 22: Invalid argument
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
oleg120-server: LNET unconfigure error 22: Invalid argument
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg120-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
oleg120-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg120-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Started lustre-OST0000
oleg120-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg120-client.virtnet:  -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-mdt1_2
Target:     fs14334:MDT0001
Index:      1
Lustre FS:  fs14334
Mount type: ldiskfs
Flags:      0x61
              (MDT first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: mgsnode=192.168.201.120@tcp:192.168.201.120@tcp sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs14334:MDT0001
	kilobytes     200000
	options       -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs14334:MDT0001 -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-mdt1_2
Started fs14334-MDT0001
mdt.fs14334-MDT0001.recovery_status=status: INACTIVE
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs3277:OST0000
Index:      0
Lustre FS:  fs3277
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.201.120@tcp:192.168.201.120@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs3277:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs3277:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs3277-OST0000-super.width=65536
oleg120-server: oleg120-server.virtnet: executing set_default_debug -1 all
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs3277-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
obdfilter.fs3277-OST0000.recovery_status=status: INACTIVE
allow_register auto-arm 24h window test completed successfully
Cleaning up test 157f...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg120-server
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg120-server
umount lustre on /mnt/lustre.....
Stopping client oleg120-client.virtnet /mnt/lustre (opts:)
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg120-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg120-server
stop mds service on oleg120-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg120-server
unloading modules via unload_modules_local on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing unload_modules_local
oleg120-server: kmemleak disabled
modules unloaded.
PASS 157f (270s)
Stopping clients: oleg120-client.virtnet /mnt/lustre (opts:-f)
Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:-f)
pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2
oleg120-server: oleg120-server.virtnet: executing set_hostid
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg120-server'
oleg120-server: oleg120-server.virtnet: executing load_modules_local
oleg120-server: kmemleak disabled
oleg120-server: Loading modules from /home/green/git/lustre-release/lustre
oleg120-server: detected 4 online CPUs by sysfs
oleg120-server: MODOPTS_LIBCFS=
oleg120-server: Force libcfs to create 2 CPU partitions
oleg120-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg120-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg120-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg120-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Formatting mgs, mds, osts
Format mds1: /dev/mapper/mds1_flakey
Format ost1: /dev/mapper/ost1_flakey
Format ost2: /dev/mapper/ost2_flakey
== conf-sanity test complete, duration 2242 sec ========== 06:49:34 (1789382974)
=== conf-sanity: start cleanup 06:49:36 (1789382976) ===
=== conf-sanity: finish cleanup 06:49:40 (1789382980) ===