-----============= acceptance-small: conf-sanity ============----- Mon Sep 14 06:10:44 EDT 2026
MGS_OS_ID_LIKE=rhel centos fedora rocky
MGS_OS_VERSION_ID=8.10
MGS_OS_ID=rocky
MGS_OS_VERSION_CODE=134873088
MDS1_OS_VERSION_ID=8.10
MDS1_OS_VERSION_CODE=134873088
MDS1_OS_ID_LIKE=rhel centos fedora rocky
MDS1_OS_ID=rocky
OST1_OS_VERSION_CODE=134873088
OST1_OS_ID_LIKE=rhel centos fedora rocky
OST1_OS_VERSION_ID=8.10
OST1_OS_ID=rocky
CLIENT_OS_ID=rocky
CLIENT_OS_VERSION_CODE=134873088
CLIENT_OS_VERSION_ID=8.10
CLIENT_OS_ID_LIKE=rhel centos fedora rocky
oleg335-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex
- see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683432 >= 34553653) for LU-16938, go 81
- see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683432 >= 34554143) for LU-18348, go 32b 32c
- see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683432 >= 34617107) for LU-17920, go 123ae 123ag
- see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683432 >= 34681950) for LU-19740, go 115
excepting tests: 32newtarball
skipping tests SLOW=no: 45 69 106 111 114
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:-f)
Stopping client oleg335-client.virtnet /mnt/lustre opts:-f
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:-f)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg335-server
oleg335-server: oleg335-server.virtnet: executing set_hostid
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Formatting mgs, mds, osts
Format mds1: /dev/mapper/mds1_flakey
Format ost1: /dev/mapper/ost1_flakey
Format ost2: /dev/mapper/ost2_flakey
start mds service on oleg335-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Commit the device label on /dev/mapper/mds1_flakey
Started lustre-MDT0000
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg335-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Commit the device label on /dev/mapper/ost1_flakey
Started lustre-OST0000
oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
oleg335-server: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50
oleg335-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec
stop ost1 service on oleg335-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server
stop mds service on oleg335-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server

== conf-sanity test 157a: test allow_register for MDT registration and MGS restart ========================================================== 06:13:37 (1789380817)
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:)
start mds service on oleg335-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg335-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-OST0000
oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg335-client.virtnet:  -o user_xattr,flock 192.168.203.135@tcp:/lustre /mnt/lustre
allow_register=unlimited
allow_register=0
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-mdt1_2
Target:     fs17460:MDT0001
Index:      1
Lustre FS:  fs17460
Mount type: ldiskfs
Flags:      0x61
              (MDT first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: mgsnode=192.168.203.135@tcp:192.168.203.135@tcp sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs17460:MDT0001
	kilobytes     200000
	options       -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs17460:MDT0001 -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg335-server: mount.lustre: mount /dev/mapper/fs2mds_flakey at /mnt/lustre-fs2mds failed: Permission denied
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 13
Start of /dev/mapper/fs2mds_flakey on fs2mds failed 13
allow_register=unlimited
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-mdt1_2
Started fs17460-MDT0001
mdt.fs17460-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg335-server
allow_register=0
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started fs17460-MDT0001
mdt.fs17460-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg335-server
stop mds service on oleg335-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
start mds service on oleg335-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
affected facets: mgs
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started fs17460-MDT0001
mdt.fs17460-MDT0001.recovery_status=status: INACTIVE
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg335-server
allow_register functionality test completed successfully
Cleaning up test 157a...
umount lustre on /mnt/lustre.....
Stopping client oleg335-client.virtnet /mnt/lustre (opts:)
oleg335-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg335-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server
stop mds service on oleg335-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
PASS 157a (269s)

== conf-sanity test 157b: verify allow_register (block new OSTs, allow existing) ========================================================== 06:18:06 (1789381086)
start mds service on oleg335-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg335-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg335-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-OST0000
oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg335-client.virtnet:  -o user_xattr,flock 192.168.203.135@tcp:/lustre /mnt/lustre
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs4300:OST0000
Index:      0
Lustre FS:  fs4300
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.135@tcp:192.168.203.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs4300:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs4300:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs4300-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs4300-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
obdfilter.fs4300-OST0000.recovery_status=status: INACTIVE
allow_register=0
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg335-server
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs4300-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started fs4300-OST0000
obdfilter.fs4300-OST0000.recovery_status=status: INACTIVE
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost2_2
Target:     fs4300:OST0001
Index:      1
Lustre FS:  fs4300
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.135@tcp:192.168.203.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs4300:OST0001
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs4300:OST0001 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
oleg335-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 13
oleg335-server: error: set_param: param_path 'seq/cli-fs4300:OST0001-super/width': No such file or directory
oleg335-server: error: set_param: setting 'seq/cli-fs4300:OST0001-super/width'='65536': No such file or directory
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13
allow_register=unlimited
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs4300-OST0001-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost2_2
Started fs4300-OST0001
/mnt/lustre-fs3ost: 83.1 MiB (87142400 bytes) trimmed
obdfilter.fs4300-OST0000.recovery_status=status: INACTIVE
obdfilter.fs4300-OST0001.recovery_status=status: INACTIVE
allow_register=0
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg335-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg335-server
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs4300-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started fs4300-OST0000
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs4300-OST0001-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started fs4300-OST0001
Passed: Existing targets can re-register, new targets blocked
Cleaning up test 157b...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg335-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg335-server
umount lustre on /mnt/lustre.....
Stopping client oleg335-client.virtnet /mnt/lustre (opts:)
oleg335-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg335-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server
stop mds service on oleg335-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
PASS 157b (348s)

== conf-sanity test 157c: test allow_register timeout functionality ========================================================== 06:23:54 (1789381434)
start mds service on oleg335-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg335-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg335-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-OST0000
oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg335-client.virtnet:  -o user_xattr,flock 192.168.203.135@tcp:/lustre /mnt/lustre
allow_register=unlimited
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs10217:OST0000
Index:      0
Lustre FS:  fs10217
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.135@tcp:192.168.203.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs10217:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs10217:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs10217-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs10217-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
allow_register=60s
obdfilter.fs10217-OST0000.recovery_status=status: INACTIVE
Waiting for allow_register timeout to expire
Waiting 66s for '1'
Updated after 57s: want '1' got '1'
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost2_2
Target:     fs10217:OST0001
Index:      1
Lustre FS:  fs10217
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.135@tcp:192.168.203.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs10217:OST0001
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs10217:OST0001 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
oleg335-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 13
oleg335-server: error: set_param: param_path 'seq/cli-fs10217:OST0001-super/width': No such file or directory
oleg335-server: error: set_param: setting 'seq/cli-fs10217:OST0001-super/width'='65536': No such file or directory
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13
allow_register=unlimited
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
Start fs3ost: mount -t lustre -o localrecov  /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost
seq.cli-fs10217-OST0001-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost2_2
Started fs10217-OST0001
/mnt/lustre-fs3ost: 83.1 MiB (87146496 bytes) trimmed
obdfilter.fs10217-OST0000.recovery_status=status: INACTIVE
obdfilter.fs10217-OST0001.recovery_status=status: INACTIVE
Cleaning up test 157c...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg335-server
Stopping /mnt/lustre-fs3ost (opts:-f) on oleg335-server
umount lustre on /mnt/lustre.....
Stopping client oleg335-client.virtnet /mnt/lustre (opts:)
oleg335-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg335-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server
stop mds service on oleg335-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
PASS 157c (315s)

== conf-sanity test 157d: test allow_register -P persistence and legacy 1->unlimited ========================================================== 06:29:09 (1789381749)
start mds service on oleg335-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg335-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg335-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-OST0000
oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg335-client.virtnet:  -o user_xattr,flock 192.168.203.135@tcp:/lustre /mnt/lustre
Waiting 90s for '0'
Updated after 8s: want '0' got '0'
oleg335-server: error: executing set_param: Invalid argument
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 22
oleg335-server: error: executing set_param: Invalid argument
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 22
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:)
Stopping client oleg335-client.virtnet /mnt/lustre opts:
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg335-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
affected facets: mgs
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:)
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg335-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
affected facets: mgs
allow_register persistence and legacy 1->unlimited OK
Cleaning up test 157d...
umount lustre on /mnt/lustre.....
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg335-server
stop mds service on oleg335-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
PASS 157d (317s)

== conf-sanity test 157e: test allow_register mgs module parameter boot-time enforcement ========================================================== 06:34:26 (1789382066)
start mds service on oleg335-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg335-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-OST0000
oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg335-client.virtnet:  -o user_xattr,flock 192.168.203.135@tcp:/lustre /mnt/lustre
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:)
Stopping client oleg335-client.virtnet /mnt/lustre opts:
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg335-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
allow_register=60s
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg335-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg335-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
affected facets: mgs
mgs allow_register module parameter OK
Cleaning up test 157e...
oleg335-server: error: executing set_param: No such file or directory
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
umount lustre on /mnt/lustre.....
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg335-server
stop mds service on oleg335-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
PASS 157e (355s)

== conf-sanity test 157f: test allow_register auto-arm default 24-hour window ========================================================== 06:40:21 (1789382421)
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:)
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:)
LNET unconfigure error 22: Invalid argument
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
oleg335-server: LNET unconfigure error 22: Invalid argument
modules unloaded.
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
start mds service on oleg335-server
Start mds1: mount -t lustre -o localrecov  /dev/mapper/mds1_flakey /mnt/lustre-mds1
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-MDT0000
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg335-server
Start ost1: mount -t lustre -o localrecov  /dev/mapper/ost1_flakey /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Started lustre-OST0000
oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg335-client.virtnet:  -o user_xattr,flock 192.168.203.135@tcp:/lustre /mnt/lustre
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-mdt1_2
Target:     fs6086:MDT0001
Index:      1
Lustre FS:  fs6086
Mount type: ldiskfs
Flags:      0x61
              (MDT first_time update )
Persistent mount opts: user_xattr,errors=remount-ro
Parameters: mgsnode=192.168.203.135@tcp:192.168.203.135@tcp sys.timeout=20 mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity

formatting backing filesystem ldiskfs on /dev/loop0
	target name   fs6086:MDT0001
	kilobytes     200000
	options       -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs6086:MDT0001 -b 4096  -I 1024 -i 2560 -q -O uninit_bg,extents,dirdata,dir_nlink,quota,project,huge_file,ea_inode,large_dir,^fast_commit,flex_bg -E lazy_itable_init,lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs2mds: mount -t lustre -o localrecov  /dev/mapper/fs2mds_flakey /mnt/lustre-fs2mds
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-mdt1_2
Started fs6086-MDT0001
mdt.fs6086-MDT0001.recovery_status=status: INACTIVE
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1

   Permanent disk data:
Device:     /tmp/lustre-ost1_2
Target:     fs18749:OST0000
Index:      0
Lustre FS:  fs18749
Mount type: ldiskfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: errors=remount-ro
Parameters: mgsnode=192.168.203.135@tcp:192.168.203.135@tcp sys.timeout=20

formatting backing filesystem ldiskfs on /dev/loop1
	target name   fs18749:OST0000
	kilobytes     200000
	options       -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F
mkfs_cmd = mke2fs -j -b 4096 -L fs18749:OST0000 -b 4096  -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k
/home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1890: OST_INDICES: bad array subscript
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Start fs2ost: mount -t lustre -o localrecov  /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost
seq.cli-fs18749-OST0000-super.width=65536
oleg335-server: oleg335-server.virtnet: executing set_default_debug -1 all
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 1
Commit the device label on /tmp/lustre-ost1_2
Started fs18749-OST0000
/mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed
obdfilter.fs18749-OST0000.recovery_status=status: INACTIVE
allow_register auto-arm 24h window test completed successfully
Cleaning up test 157f...
Stopping /mnt/lustre-fs2ost (opts:-f) on oleg335-server
Stopping /mnt/lustre-fs2mds (opts:-f) on oleg335-server
umount lustre on /mnt/lustre.....
Stopping client oleg335-client.virtnet /mnt/lustre (opts:)
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 123
allow_register=unlimited
stop ost1 service on oleg335-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg335-server
stop mds service on oleg335-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg335-server
unloading modules via unload_modules_local on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing unload_modules_local
oleg335-server: kmemleak disabled
modules unloaded.
PASS 157f (254s)
Stopping clients: oleg335-client.virtnet /mnt/lustre (opts:-f)
Stopping clients: oleg335-client.virtnet /mnt/lustre2 (opts:-f)
pdsh@oleg335-client: oleg335-server: ssh exited with exit code 2
oleg335-server: oleg335-server.virtnet: executing set_hostid
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
../libcfs/libcfs/libcfs options: 'cpu_npartitions=2'
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1931560 Sep 14 04:14 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg335-server'
oleg335-server: oleg335-server.virtnet: executing load_modules_local
oleg335-server: kmemleak disabled
oleg335-server: Loading modules from /home/green/git/lustre-release/lustre
oleg335-server: detected 4 online CPUs by sysfs
oleg335-server: MODOPTS_LIBCFS=
oleg335-server: Force libcfs to create 2 CPU partitions
oleg335-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg335-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg335-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg335-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Formatting mgs, mds, osts
Format mds1: /dev/mapper/mds1_flakey
Format ost1: /dev/mapper/ost1_flakey
Format ost2: /dev/mapper/ost2_flakey
== conf-sanity test complete, duration 2090 sec ========== 06:45:36 (1789382736)
=== conf-sanity: start cleanup 06:45:38 (1789382738) ===
=== conf-sanity: finish cleanup 06:45:42 (1789382742) ===