-----============= acceptance-small: conf-sanity ============----- Fri Sep  4 07:15:01 EDT 2026
MGS_OS_ID_LIKE=rhel centos fedora rocky
MGS_OS_VERSION_ID=8.10
MGS_OS_ID=rocky
MGS_OS_VERSION_CODE=134873088
MDS1_OS_VERSION_ID=8.10
MDS1_OS_VERSION_CODE=134873088
MDS1_OS_ID_LIKE=rhel centos fedora rocky
MDS1_OS_ID=rocky
OST1_OS_VERSION_CODE=134873088
OST1_OS_ID_LIKE=rhel centos fedora rocky
OST1_OS_VERSION_ID=8.10
OST1_OS_ID=rocky
CLIENT_OS_ID=rocky
CLIENT_OS_VERSION_CODE=134873088
CLIENT_OS_VERSION_ID=8.10
CLIENT_OS_ID_LIKE=rhel centos fedora rocky
oleg205-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex
- see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683394 >= 34553653) for LU-16938, go 81
- see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683394 >= 34554143) for LU-18348, go 32b 32c
- see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683394 >= 34617107) for LU-17920, go 123ae 123ag
- see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683394 >= 34681950) for LU-19740, go 115
excepting tests: 21e 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a
skipping tests SLOW=no: 45 69 106 111 114
Stopping clients: oleg205-client.virtnet /mnt/lustre (opts:-f)
Stopping client oleg205-client.virtnet /mnt/lustre opts:-f
Stopping clients: oleg205-client.virtnet /mnt/lustre2 (opts:-f)
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
Stopping /mnt/lustre-ost2 (opts:-f) on oleg205-server
oleg205-server: oleg205-server.virtnet: executing set_hostid
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Formatting mgs, mds, osts
Format mds1: lustre-mdt1/mdt1
Format ost1: lustre-ost1/ost1
Format ost2: lustre-ost2/ost2
start mds service on oleg205-server
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Commit the device label on lustre-mdt1/mdt1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Commit the device label on lustre-ost1/ost1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
oleg205-server: oleg205-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50
oleg205-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server

== conf-sanity test 0: single mount setup ================ 07:18:01 (1788520681)
start mds service on oleg205-server
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
setup single mount lustre success
umount lustre on /mnt/lustre.....
Stopping client oleg205-client.virtnet /mnt/lustre (opts:)
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
PASS 0 (85s)

== conf-sanity test 1: start up ost twice (should return errors) ========================================================== 07:19:26 (1788520766)
start mds service on oleg205-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
start ost second time...
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
oleg205-server: mount.lustre: according to /etc/mtab lustre-ost1/ost1 is already mounted on /mnt/lustre-ost1
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 17
seq.cli-lustre-OST0000-super.width=65536
Start of lustre-ost1/ost1 on ost1 failed 17
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
setup single mount lustre success
umount lustre on /mnt/lustre.....
Stopping client oleg205-client.virtnet /mnt/lustre (opts:)
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
PASS 1 (118s)

== conf-sanity test 2: start up mds twice (should return err) ========================================================== 07:21:25 (1788520885)
start mds service on oleg205-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start mds second time..
start mds service on oleg205-server
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: mount.lustre: according to /etc/mtab lustre-mdt1/mdt1 is already mounted on /mnt/lustre-mds1
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 17
Start of lustre-mdt1/mdt1 on mds1 failed 17
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
setup single mount lustre success
umount lustre on /mnt/lustre.....
Stopping client oleg205-client.virtnet /mnt/lustre (opts:)
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
PASS 2 (139s)

== conf-sanity test 3: mount client twice (should return err) ========================================================== 07:23:43 (1788521023)
start mds service on oleg205-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
mount.lustre: according to /etc/mtab 192.168.202.105@tcp:/lustre is already mounted on /mnt/lustre
setup single mount lustre success
umount lustre on /mnt/lustre.....
Stopping client oleg205-client.virtnet /mnt/lustre (opts:)
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
PASS 3 (122s)

== conf-sanity test 4: force cleanup ost, then cleanup === 07:25:45 (1788521145)
start mds service on oleg205-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
umount lustre on /mnt/lustre.....
Stopping client oleg205-client.virtnet /mnt/lustre (opts:-f)
stop ost1 service on oleg205-server
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
PASS 4 (140s)

== conf-sanity test 5a: force cleanup mds, then cleanup == 07:28:05 (1788521285)
start mds service on oleg205-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
                     USER        PID ACCESS COMMAND
/mnt/lustre:         root     kernel mount /mnt/lustre
/mnt/lustre is in use by user space process.
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
killing umount
waiting for umount to finish
/home/green/git/lustre-release/lustre/tests/conf-sanity.sh: line 149: 27890 Terminated              $UMOUNT -f $MOUNT
manual umount lustre on /mnt/lustre....
umount: /mnt/lustre: not mounted.
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
stop mds service on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
manual umount lustre on /mnt/lustre....
umount: /mnt/lustre: not mounted.
/etc/mtab updated in 0 secs
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
PASS 5a (133s)

== conf-sanity test 5b: Try to start a client with no MGS (should return errs) ========================================================== 07:30:19 (1788521419)
start ost1 service on oleg205-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
mount.lustre: mount 192.168.202.105@tcp:/lustre at /mnt/lustre failed: Connection timed out
umount lustre on /mnt/lustre.....
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
stop mds service on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
PASS 5b (295s)

== conf-sanity test 5c: cleanup after failed mount (bug 2712) (should return errs) ========================================================== 07:35:14 (1788521714)
start mds service on oleg205-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount wrong.lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/wrong.lustre /mnt/lustre
mount.lustre: mount 192.168.202.105@tcp:/wrong.lustre at /mnt/lustre failed: File name too long
umount lustre on /mnt/lustre.....
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
PASS 5c (120s)

== conf-sanity test 5d: mount with ost down ============== 07:37:14 (1788521834)
start ost1 service on oleg205-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
start mds service on oleg205-server
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
umount lustre on /mnt/lustre.....
Stopping client oleg205-client.virtnet /mnt/lustre (opts:-f)
stop ost1 service on oleg205-server
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
PASS 5d (194s)

== conf-sanity test 5e: delayed connect, don't crash (bug 10268) ========================================================== 07:40:28 (1788522028)
start mds service on oleg205-server
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
fail_loc=0x80000506
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
umount lustre on /mnt/lustre.....
Stopping client oleg205-client.virtnet /mnt/lustre (opts:)
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server
stop mds service on oleg205-server
Stopping /mnt/lustre-mds1 (opts:-f) on oleg205-server
unloading modules via unload_modules_local on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing unload_modules_local
oleg205-server: kmemleak disabled
modules unloaded.
PASS 
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-client: ssh exited with exit code 2
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
PASS 5e (135s)

== conf-sanity test 5f: mds down, cleanup after failed mount (bug 2712) ========================================================== 07:42:43 (1788522163)

 SKIP: conf-sanity test_5f needs separate mgs and mds
SKIP 5f (5s)

== conf-sanity test 5g: handle missing debugfs =========== 07:42:48 (1788522168)
Loading modules from /home/green/git/lustre-release/lustre
detected 4 online CPUs by sysfs
MODOPTS_LIBCFS=
Force libcfs to create 2 CPU partitions
ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
quota/lquota options: 'hash_lqs_cur_bits=3'
mdt/mdt options: 'mdt_enable_flr_ec=1'
-rwxr-xr-x. 1 green green 1921056 Sep  3 15:04 /home/green/git/lustre-release/lustre/utils/mount.lustre
-rwxr-xr-x. 1 root  root       12 Jan 31  2019 /usr/sbin/mount.lustre
mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre
loading modules on: 'oleg205-server'
oleg205-server: oleg205-server.virtnet: executing load_modules_local
oleg205-server: kmemleak disabled
oleg205-server: Loading modules from /home/green/git/lustre-release/lustre
oleg205-server: detected 4 online CPUs by sysfs
oleg205-server: MODOPTS_LIBCFS=
oleg205-server: Force libcfs to create 2 CPU partitions
oleg205-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1'
oleg205-server: quota/lquota options: 'hash_lqs_cur_bits=3'
oleg205-server: mdt/mdt options: 'mdt_enable_flr_ec=1'
oleg205-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt
none /sys/kernel/debug debugfs rw,relatime 0 0
PASS 5g (33s)

== conf-sanity test 5h: start mdt failure at mdt_fs_setup() ========================================================== 07:43:21 (1788522201)
start mds service on oleg205-server
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
oleg205-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
start ost1 service on oleg205-server
Start ost1: mount -t lustre -o localrecov  lustre-ost1/ost1 /mnt/lustre-ost1
seq.cli-lustre-OST0000-super.width=65536
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-OST0000
oleg205-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0
mount lustre  on /mnt/lustre.....
Starting client: oleg205-client.virtnet:  -o user_xattr,flock 192.168.202.105@tcp:/lustre /mnt/lustre
Stopping /mnt/lustre-mds1 (opts:) on oleg205-server
fail_loc=0x80000135
start mds service on oleg205-server
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: mount.lustre: mount lustre-mdt1/mdt1 at /mnt/lustre-mds1 failed: No such file or directory
oleg205-server: Is the MGS specification correct?
oleg205-server: Is the filesystem name correct?
oleg205-server: If upgrading, is the copied client log valid? (see upgrade docs)
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 2
Start of lustre-mdt1/mdt1 on mds1 failed 2
start mds service on oleg205-server
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg205-server: oleg205-server.virtnet: executing set_default_debug -1 all
pdsh@oleg205-client: oleg205-server: ssh exited with exit code 1
Started lustre-MDT0000
umount lustre on /mnt/lustre.....
Stopping client oleg205-client.virtnet /mnt/lustre (opts:)
stop ost1 service on oleg205-server
Stopping /mnt/lustre-ost1 (opts:-f) on oleg205-server