-----============= acceptance-small: conf-sanity ============----- Mon Sep 14 06:27:34 EDT 2026
MGS_OS_ID_LIKE=rhel centos fedora rocky
MGS_OS_VERSION_ID=8.10
MGS_OS_ID=rocky
MGS_OS_VERSION_CODE=134873088
MDS1_OS_VERSION_ID=8.10
MDS1_OS_VERSION_CODE=134873088
MDS1_OS_ID_LIKE=rhel centos fedora rocky
MDS1_OS_ID=rocky
OST1_OS_VERSION_CODE=134873088
OST1_OS_ID_LIKE=rhel centos fedora rocky
OST1_OS_VERSION_ID=8.10
OST1_OS_ID=rocky
CLIENT_OS_ID=rocky
CLIENT_OS_VERSION_CODE=134873088
CLIENT_OS_VERSION_ID=8.10
CLIENT_OS_ID_LIKE=rhel centos fedora rocky
oleg331-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex
- see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683432 >= 34553653) for LU-16938, go 81
- see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683432 >= 34554143) for LU-18348, go 32b 32c
- see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683432 >= 34617107) for LU-17920, go 123ae 123ag
- see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683432 >= 34681950) for LU-19740, go 115
excepting tests: 32newtarball
skipping tests SLOW=no: 45 69 106 111 114
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:-f)
Stopping client oleg331-client.virtnet /mnt/lustre opts:-f
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:-f)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg331-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg331-server
oleg331-server: oleg331-server.virtnet: executing set_hostid
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Formatting mgs, mds, osts
Format mds1: /dev/mapper/mds1_flakey
Format ost1: /dev/mapper/ost1_flakey
Format ost2: /dev/mapper/ost2_flakey
start mds service on oleg331-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Commit the device label on /dev/mapper/mds1_flakey
Started lustre-MDT0000
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
oleg331-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg331-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Commit the device label on /dev/mapper/ost1_flakey
Started lustre-OST0000
oleg331-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
oleg331-server: oleg331-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50
oleg331-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec
stop ost1 service on oleg331-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg331-server
stop mds service on oleg331-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server

== conf-sanity test 157a: test allow_register for MDT registration and MGS restart ========================================================== 06:30:27 (1789381827)
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:)
start mds service on oleg331-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
oleg331-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg331-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-OST0000
oleg331-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg331-client.virtnet:  -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre
allow_register=unlimited
allow_register=0
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-mdt1_2
Target:     fs21854:MDT0001
Index:      1
Lustre FS:  fs21854
Mount type: ldiskfs
Flags:      0x61
              (MDT first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: mgsnode=192.168.203.131@tcp:192.168.203.131@tcp sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs21854:MDT0001
	kilobytes     200000
	options       -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs21854:MDT0001 -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg331-server: mount.lustre: mount /dev/mapper/fs2mds_flakey at /mnt/lustre-fs2mds failed: Permission denied
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 13
Start of /dev/mapper/fs2mds_flakey on fs2mds failed 13
allow_register=unlimited
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-mdt1_2
Started fs21854-MDT0001
mdt.fs21854-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg331-server
allow_register=0
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started fs21854-MDT0001
mdt.fs21854-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg331-server
stop mds service on oleg331-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
start mds service on oleg331-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
affected facets: mgs
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started fs21854-MDT0001
mdt.fs21854-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg331-server
allow_register functionality test completed successfully
Cleaning up test 157a...
umount lustre on /mnt/lustre.....
Stopping client oleg331-client.virtnet /mnt/lustre (opts:)
oleg331-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg331-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg331-server
stop mds service on oleg331-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
PASS 157a (266s)

== conf-sanity test 157b: verify allow_register (block new OSTs, allow existing) ========================================================== 06:34:53 (1789382093)
start mds service on oleg331-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg331-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
oleg331-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg331-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-OST0000
oleg331-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg331-client.virtnet:  -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs31795:OST0000
Index:      0
Lustre FS:  fs31795
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.131@tcp:192.168.203.131@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs31795:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs31795:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs31795-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs31795-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
obdfilter.fs31795-OST0000.recovery_status=status: INACTIVE
allow_register=0
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg331-server
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs31795-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started fs31795-OST0000
obdfilter.fs31795-OST0000.recovery_status=status: INACTIVE
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost2_2
Target:     fs31795:OST0001
Index:      1
Lustre FS:  fs31795
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.131@tcp:192.168.203.131@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs31795:OST0001
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs31795:OST0001 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
oleg331-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 13
oleg331-server: error: set_param: param_path 'seq/cli-fs31795:OST0001-super/width': No such file or directory
oleg331-server: error: set_param: setting 'seq/cli-fs31795:OST0001-super/width'='65536': No such file or directory
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13
allow_register=unlimited
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs31795-OST0001-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost2_2
Started fs31795-OST0001
/mnt/lustre-fs3ost: 83.1 MiB (87146496 bytes) trimmed
obdfilter.fs31795-OST0000.recovery_status=status: INACTIVE
obdfilter.fs31795-OST0001.recovery_status=status: INACTIVE
allow_register=0
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg331-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg331-server
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs31795-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started fs31795-OST0000
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs31795-OST0001-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started fs31795-OST0001
Passed: Existing targets can re-register, new targets blocked
Cleaning up test 157b...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg331-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg331-server
umount lustre on /mnt/lustre.....
Stopping client oleg331-client.virtnet /mnt/lustre (opts:)
oleg331-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg331-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg331-server
stop mds service on oleg331-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
PASS 157b (325s)

== conf-sanity test 157c: test allow_register timeout functionality ========================================================== 06:40:18 (1789382418)
start mds service on oleg331-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg331-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
oleg331-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg331-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-OST0000
oleg331-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg331-client.virtnet:  -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre
allow_register=unlimited
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs17595:OST0000
Index:      0
Lustre FS:  fs17595
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.131@tcp:192.168.203.131@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs17595:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs17595:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs17595-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs17595-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
allow_register=60s
obdfilter.fs17595-OST0000.recovery_status=status: INACTIVE
Waiting for allow_register timeout to expire
Waiting 67s for '1'
Waiting 57s for '1'
Waiting 47s for '1'
Waiting 37s for '1'
Waiting 27s for '1'
Waiting 17s for '1'
Updated after 58s: want '1' got '1'
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost2_2
Target:     fs17595:OST0001
Index:      1
Lustre FS:  fs17595
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.131@tcp:192.168.203.131@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs17595:OST0001
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs17595:OST0001 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
oleg331-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 13
oleg331-server: error: set_param: param_path 'seq/cli-fs17595:OST0001-super/width': No such file or directory
oleg331-server: error: set_param: setting 'seq/cli-fs17595:OST0001-super/width'='65536': No such file or directory
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13
allow_register=unlimited
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs17595-OST0001-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost2_2
Started fs17595-OST0001
/mnt/lustre-fs3ost: 83.1 MiB (87146496 bytes) trimmed
obdfilter.fs17595-OST0000.recovery_status=status: INACTIVE
obdfilter.fs17595-OST0001.recovery_status=status: INACTIVE
Cleaning up test 157c...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg331-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg331-server
umount lustre on /mnt/lustre.....
Stopping client oleg331-client.virtnet /mnt/lustre (opts:)
oleg331-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg331-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg331-server
stop mds service on oleg331-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
PASS 157c (292s)

== conf-sanity test 157d: test allow_register -P persistence and legacy 1->unlimited ========================================================== 06:45:10 (1789382710)
start mds service on oleg331-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg331-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
oleg331-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg331-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-OST0000
oleg331-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg331-client.virtnet:  -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre
Waiting 90s for '0'
Updated after 7s: want '0' got '0'
oleg331-server: error: executing set_param: Invalid argument
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22
oleg331-server: error: executing set_param: Invalid argument
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 22
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:)
Stopping client oleg331-client.virtnet /mnt/lustre opts:
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg331-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
affected facets: mgs
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:)
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg331-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
affected facets: mgs
allow_register persistence and legacy 1->unlimited OK
Cleaning up test 157d...
umount lustre on /mnt/lustre.....
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg331-server
stop mds service on oleg331-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
PASS 157d (310s)

== conf-sanity test 157e: test allow_register mgs module parameter boot-time enforcement ========================================================== 06:50:20 (1789383020)
start mds service on oleg331-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
oleg331-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg331-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-OST0000
oleg331-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg331-client.virtnet:  -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:)
Stopping client oleg331-client.virtnet /mnt/lustre opts:
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg331-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
allow_register=60s
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg331-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg331-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
mgs allow_register module parameter OK
Cleaning up test 157e...
oleg331-server: error: executing set_param: No such file or directory
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
umount lustre on /mnt/lustre.....
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg331-server
stop mds service on oleg331-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
PASS 157e (354s)

== conf-sanity test 157f: test allow_register auto-arm default 24-hour window ========================================================== 06:56:14 (1789383374)
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:)
LNET unconfigure error 22: Invalid argument
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
oleg331-server: LNET unconfigure error 22: Invalid argument
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg331-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
oleg331-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg331-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Started lustre-OST0000
oleg331-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg331-client.virtnet:  -o user_xattr,flock 192.168.203.131@tcp:/lustre /mnt/lustre
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-mdt1_2
Target:     fs20113:MDT0001
Index:      1
Lustre FS:  fs20113
Mount type: ldiskfs
Flags:      0x61
              (MDT first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: mgsnode=192.168.203.131@tcp:192.168.203.131@tcp sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs20113:MDT0001
	kilobytes     200000
	options       -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs20113:MDT0001 -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-mdt1_2
Started fs20113-MDT0001
mdt.fs20113-MDT0001.recovery_status=status: INACTIVE
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs13811:OST0000
Index:      0
Lustre FS:  fs13811
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.131@tcp:192.168.203.131@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs13811:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs13811:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs13811-OST0000-super.width=65536
oleg331-server: oleg331-server.virtnet: executing set_default_debug -1 all
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs13811-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
obdfilter.fs13811-OST0000.recovery_status=status: INACTIVE
allow_register auto-arm 24h window test completed successfully
Cleaning up test 157f...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg331-server
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg331-server
umount lustre on /mnt/lustre.....
Stopping client oleg331-client.virtnet /mnt/lustre (opts:)
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg331-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg331-server
stop mds service on oleg331-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg331-server
unloading modules via unload_modules_local on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing unload_modules_local
oleg331-server: kmemleak disabled
modules unloaded.
PASS 157f (257s)
Stopping clients: oleg331-client.virtnet /mnt/lustre (opts:-f)
Stopping clients: oleg331-client.virtnet /mnt/lustre2 (opts:-f)
pdsh@oleg331-client: oleg331-server: ssh exited with exit code 2
oleg331-server: oleg331-server.virtnet: executing set_hostid
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg331-server'
oleg331-server: oleg331-server.virtnet: executing load_modules_local
oleg331-server: kmemleak disabled
oleg331-server: Loading modules from /home/green/git/lustre-release/lustre
oleg331-server: detected 4 online CPUs by sysfs
oleg331-server: MODOPTS_LIBCFS=
oleg331-server: Force libcfs to create 2 CPU partitions
oleg331-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg331-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg331-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg331-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Formatting mgs, mds, osts
Format mds1: /dev/mapper/mds1_flakey
Format ost1: /dev/mapper/ost1_flakey
Format ost2: /dev/mapper/ost2_flakey
== conf-sanity test complete, duration 2037 sec ========== 07:01:32 (1789383692)
=== conf-sanity: start cleanup 07:01:34 (1789383694) ===
=== conf-sanity: finish cleanup 07:01:38 (1789383698) ===