-----============= acceptance-small: conf-sanity ============----- Thu Jan 22 04:20:38 EST 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg419-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 21e 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a skipping tests SLOW=no: 45 69 106 111 114 Stopping clients: oleg419-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg419-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg419-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg419-server oleg419-server: oleg419-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing load_modules_local oleg419-server: Loading modules from /home/green/git/lustre-release/lustre oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg419-server: detected 4 online CPUs by sysfs oleg419-server: MODOPTS_LIBCFS= oleg419-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg419-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg419-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg419-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg419-server: oleg419-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg419-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server stop mds service on oleg419-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg419-server == conf-sanity test 0: single mount setup ================ 04:24:01 (1769073841) start mds service on oleg419-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg419-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg419-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg419-client.virtnet: -o user_xattr,flock 192.168.204.119@tcp:/lustre /mnt/lustre setup single mount lustre success umount lustre on /mnt/lustre..... Stopping client oleg419-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server stop mds service on oleg419-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg419-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing unload_modules_local oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 PASS 0 (92s) == conf-sanity test 1: start up ost twice (should return errors) ========================================================== 04:25:33 (1769073933) start mds service on oleg419-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing load_modules_local oleg419-server: Loading modules from /home/green/git/lustre-release/lustre oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg419-server: detected 4 online CPUs by sysfs oleg419-server: MODOPTS_LIBCFS= oleg419-server: Force libcfs to create 2 CPU partitions oleg419-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg419-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg419-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg419-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid start ost second time... start ost1 service on oleg419-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg419-server: mount.lustre: according to /etc/mtab lustre-ost1/ost1 is already mounted on /mnt/lustre-ost1 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 17 seq.cli-lustre-OST0000-super.width=65536 Start of lustre-ost1/ost1 on ost1 failed 17 mount lustre on /mnt/lustre..... Starting client: oleg419-client.virtnet: -o user_xattr,flock 192.168.204.119@tcp:/lustre /mnt/lustre setup single mount lustre success umount lustre on /mnt/lustre..... Stopping client oleg419-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server stop mds service on oleg419-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg419-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing unload_modules_local oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 PASS 1 (126s) == conf-sanity test 2: start up mds twice (should return err) ========================================================== 04:27:39 (1769074059) start mds service on oleg419-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing load_modules_local oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg419-server: Loading modules from /home/green/git/lustre-release/lustre oleg419-server: detected 4 online CPUs by sysfs oleg419-server: MODOPTS_LIBCFS= oleg419-server: Force libcfs to create 2 CPU partitions oleg419-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg419-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg419-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start mds second time.. start mds service on oleg419-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: mount.lustre: according to /etc/mtab lustre-mdt1/mdt1 is already mounted on /mnt/lustre-mds1 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 17 Start of lustre-mdt1/mdt1 on mds1 failed 17 start ost1 service on oleg419-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg419-client.virtnet: -o user_xattr,flock 192.168.204.119@tcp:/lustre /mnt/lustre setup single mount lustre success umount lustre on /mnt/lustre..... Stopping client oleg419-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server stop mds service on oleg419-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg419-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing unload_modules_local oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 PASS 2 (125s) == conf-sanity test 3: mount client twice (should return err) ========================================================== 04:29:44 (1769074184) start mds service on oleg419-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing load_modules_local oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg419-server: Loading modules from /home/green/git/lustre-release/lustre oleg419-server: detected 4 online CPUs by sysfs oleg419-server: MODOPTS_LIBCFS= oleg419-server: Force libcfs to create 2 CPU partitions oleg419-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg419-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg419-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg419-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg419-client.virtnet: -o user_xattr,flock 192.168.204.119@tcp:/lustre /mnt/lustre mount lustre on /mnt/lustre..... Starting client: oleg419-client.virtnet: -o user_xattr,flock 192.168.204.119@tcp:/lustre /mnt/lustre mount.lustre: according to /etc/mtab 192.168.204.119@tcp:/lustre is already mounted on /mnt/lustre setup single mount lustre success umount lustre on /mnt/lustre..... Stopping client oleg419-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server stop mds service on oleg419-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg419-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing unload_modules_local oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 PASS 3 (114s) == conf-sanity test 4: force cleanup ost, then cleanup === 04:31:38 (1769074298) start mds service on oleg419-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing load_modules_local oleg419-server: Loading modules from /home/green/git/lustre-release/lustre oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg419-server: detected 4 online CPUs by sysfs oleg419-server: MODOPTS_LIBCFS= oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg419-server: Force libcfs to create 2 CPU partitions oleg419-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg419-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg419-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg419-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg419-client.virtnet: -o user_xattr,flock 192.168.204.119@tcp:/lustre /mnt/lustre stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server umount lustre on /mnt/lustre..... Stopping client oleg419-client.virtnet /mnt/lustre (opts:-f) stop ost1 service on oleg419-server stop mds service on oleg419-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg419-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing unload_modules_local oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 PASS 4 (139s) == conf-sanity test 5a: force cleanup mds, then cleanup == 04:33:57 (1769074437) start mds service on oleg419-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing load_modules_local oleg419-server: Loading modules from /home/green/git/lustre-release/lustre oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg419-server: detected 4 online CPUs by sysfs oleg419-server: MODOPTS_LIBCFS= oleg419-server: Force libcfs to create 2 CPU partitions oleg419-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg419-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg419-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg419-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg419-client.virtnet: -o user_xattr,flock 192.168.204.119@tcp:/lustre /mnt/lustre USER PID ACCESS COMMAND /mnt/lustre: root kernel mount /mnt/lustre /mnt/lustre is in use by user space process. stop mds service on oleg419-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg419-server killing umount waiting for umount to finish /home/green/git/lustre-release/lustre/tests/conf-sanity.sh: line 149: 27368 Terminated $UMOUNT -f $MOUNT manual umount lustre on /mnt/lustre.... umount: /mnt/lustre: not mounted. stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server stop mds service on oleg419-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing unload_modules_local oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. manual umount lustre on /mnt/lustre.... umount: /mnt/lustre: not mounted. /etc/mtab updated in 0 secs pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 PASS 5a (123s) == conf-sanity test 5b: Try to start a client with no MGS (should return errs) ========================================================== 04:36:00 (1769074560) start ost1 service on oleg419-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing load_modules_local oleg419-server: Loading modules from /home/green/git/lustre-release/lustre oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg419-server: detected 4 online CPUs by sysfs oleg419-server: MODOPTS_LIBCFS= oleg419-server: Force libcfs to create 2 CPU partitions oleg419-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg419-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg419-client.virtnet: -o user_xattr,flock 192.168.204.119@tcp:/lustre /mnt/lustre mount.lustre: mount 192.168.204.119@tcp:/lustre at /mnt/lustre failed: Connection timed out umount lustre on /mnt/lustre..... stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server stop mds service on oleg419-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing unload_modules_local oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 PASS 5b (287s) == conf-sanity test 5c: cleanup after failed mount (bug 2712) (should return errs) ========================================================== 04:40:47 (1769074847) start mds service on oleg419-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing load_modules_local oleg419-server: Loading modules from /home/green/git/lustre-release/lustre oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg419-server: detected 4 online CPUs by sysfs oleg419-server: MODOPTS_LIBCFS= oleg419-server: Force libcfs to create 2 CPU partitions oleg419-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg419-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg419-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg419-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount wrong.lustre on /mnt/lustre..... Starting client: oleg419-client.virtnet: -o user_xattr,flock 192.168.204.119@tcp:/wrong.lustre /mnt/lustre mount.lustre: mount 192.168.204.119@tcp:/wrong.lustre at /mnt/lustre failed: File name too long umount lustre on /mnt/lustre..... stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server stop mds service on oleg419-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg419-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing unload_modules_local oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-server: ssh exited with exit code 2 pdsh@oleg419-client: oleg419-client: ssh exited with exit code 2 PASS 5c (99s) == conf-sanity test 5d: mount with ost down ============== 04:42:26 (1769074946) start ost1 service on oleg419-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg419-server' oleg419-server: oleg419-server.virtnet: executing load_modules_local oleg419-server: Loading modules from /home/green/git/lustre-release/lustre oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg419-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg419-server: detected 4 online CPUs by sysfs oleg419-server: MODOPTS_LIBCFS= oleg419-server: Force libcfs to create 2 CPU partitions oleg419-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg419-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-OST0000 oleg419-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid start mds service on oleg419-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg419-server: oleg419-server.virtnet: executing set_default_debug -1 all pdsh@oleg419-client: oleg419-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg419-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid stop ost1 service on oleg419-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg419-server