-----============= acceptance-small: conf-sanity ============----- Mon Jan 12 15:46:00 EST 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg435-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg435-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg435-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg435-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg435-server oleg435-server: oleg435-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg435-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg435-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 start mds service on oleg435-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-mdt2/mdt2 Started lustre-MDT0001 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg435-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg435-server: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg435-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg435-server: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg435-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg435-server == conf-sanity test 45: long unlink handling in ptlrpcd == 15:50:19 (1768251019) start mds service on oleg435-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg435-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg435-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg435-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg435-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg435-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg435-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg435-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing unload_modules_local oleg435-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg435-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 PASS 45 (284s) == conf-sanity test 69: replace an OST with the same index ========================================================== 15:55:03 (1768251303) start mds service on oleg435-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg435-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg435-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg435-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg435-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11840 inodes available. Want 99872. rc=0 - open/close 804 (time 1768251422.72 total 10.00 last 80.36) - open/close 1575 (time 1768251432.72 total 20.01 last 77.09) - open/close 2238 (time 1768251442.73 total 30.01 last 66.28) - open/close 2841 (time 1768251452.73 total 40.01 last 60.27) - open/close 3517 (time 1768251462.73 total 50.02 last 67.59) - open/close 4332 (time 1768251472.74 total 60.02 last 81.43) - open/close 5152 (time 1768251482.75 total 70.03 last 81.97) - open/close 5892 (time 1768251492.75 total 80.03 last 73.96) - open/close 6662 (time 1768251502.75 total 90.04 last 76.98) - open/close 7508 (time 1768251512.76 total 100.04 last 84.54) - open/close 8304 (time 1768251522.77 total 110.05 last 79.54) - open/close 9145 (time 1768251532.77 total 120.05 last 84.07) - open/close 9849 (time 1768251542.78 total 130.06 last 70.34) total: 10000 open/close in 132.17 seconds: 75.66 ops/second - unlinked 0 (time 1768251549 ; total 0 ; last 0) total: 10000 unlinks in 73 seconds: 136.986298 unlinks/second - open/close 802 (time 1768251636.61 total 10.01 last 80.10) - open/close 1615 (time 1768251646.61 total 20.01 last 81.30) - open/close 2400 (time 1768251656.62 total 30.02 last 78.44) - open/close 3268 (time 1768251666.62 total 40.02 last 86.78) - open/close 4064 (time 1768251676.64 total 50.04 last 79.50) - open/close 4905 (time 1768251686.65 total 60.05 last 83.99) - open/close 5665 (time 1768251696.66 total 70.06 last 75.92) - open/close 6393 (time 1768251706.67 total 80.07 last 72.73) - open/close 7168 (time 1768251716.69 total 90.09 last 77.32) - open/close 7854 (time 1768251726.69 total 100.09 last 68.59) - open/close 8632 (time 1768251736.71 total 110.11 last 77.65) - open/close 9467 (time 1768251746.72 total 120.12 last 83.47) total: 10000 open/close in 127.81 seconds: 78.24 ops/second - unlinked 0 (time 1768251760 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 661 (time 1768251858.83 total 10.01 last 66.06) - open/close 1385 (time 1768251868.84 total 20.01 last 72.34) - open/close 2247 (time 1768251878.84 total 30.02 last 86.18) - open/close 3096 (time 1768251888.85 total 40.02 last 84.84) - open/close 3759 (time 1768251898.86 total 50.03 last 66.27) - open/close 4418 (time 1768251908.86 total 60.04 last 65.85) - open/close 4940 (time 1768251918.87 total 70.04 last 52.19) - open/close 5668 (time 1768251928.88 total 80.05 last 72.74) - open/close 6351 (time 1768251938.89 total 90.06 last 68.19) - open/close 7190 (time 1768251948.90 total 100.07 last 83.87) - open/close 7897 (time 1768251958.90 total 110.08 last 70.65) - open/close 8568 (time 1768251968.91 total 120.08 last 67.08) - open/close 9221 (time 1768251978.92 total 130.09 last 65.22) - open/close 9963 (time 1768251988.93 total 140.10 last 74.13) total: 10000 open/close in 140.56 seconds: 71.14 ops/second - unlinked 0 (time 1768251994 ; total 0 ; last 0) total: 10000 unlinks in 78 seconds: 128.205124 unlinks/second - open/close 837 (time 1768252087.92 total 10.00 last 83.68) - open/close 1619 (time 1768252097.92 total 20.00 last 78.20) - open/close 2335 (time 1768252107.93 total 30.01 last 71.56) - open/close 3157 (time 1768252117.94 total 40.02 last 82.12) - open/close 3866 (time 1768252127.95 total 50.03 last 70.79) - open/close 4689 (time 1768252137.96 total 60.04 last 82.23) - open/close 5348 (time 1768252147.97 total 70.05 last 65.86) - open/close 6019 (time 1768252157.97 total 80.05 last 67.09) - open/close 6726 (time 1768252167.97 total 90.05 last 70.69) - open/close 7486 (time 1768252177.99 total 100.07 last 75.86) - open/close 8351 (time 1768252187.99 total 110.08 last 86.45) - open/close 9057 (time 1768252198.00 total 120.09 last 70.52) - open/close 9866 (time 1768252208.01 total 130.10 last 80.83) total: 10000 open/close in 131.53 seconds: 76.03 ops/second - unlinked 0 (time 1768252213 ; total 0 ; last 0) total: 10000 unlinks in 76 seconds: 131.578949 unlinks/second - open/close 694 (time 1768252304.88 total 10.01 last 69.36) - open/close 1374 (time 1768252314.89 total 20.01 last 67.97) - open/close 2051 (time 1768252324.89 total 30.02 last 67.65) - open/close 2768 (time 1768252334.90 total 40.02 last 71.65) - open/close 3280 (time 1768252344.92 total 50.04 last 51.13) - open/close 3875 (time 1768252354.92 total 60.04 last 59.47) - open/close 4635 (time 1768252364.93 total 70.05 last 75.95) - open/close 5545 (time 1768252374.93 total 80.05 last 90.97) - open/close 6426 (time 1768252384.94 total 90.06 last 88.05) - open/close 7314 (time 1768252394.94 total 100.06 last 88.74) - open/close 8180 (time 1768252404.96 total 110.08 last 86.49) - open/close 9082 (time 1768252414.96 total 120.08 last 90.18) - open/close 9774 (time 1768252424.96 total 130.08 last 69.16) total: 10000 open/close in 133.43 seconds: 74.95 ops/second - unlinked 0 (time 1768252433 ; total 0 ; last 0) total: 10000 unlinks in 81 seconds: 123.456787 unlinks/second - open/close 809 (time 1768252529.03 total 10.00 last 80.89) - open/close 1693 (time 1768252539.03 total 20.01 last 88.35) - open/close 2531 (time 1768252549.03 total 30.01 last 83.80) - open/close 3521 (time 1768252559.03 total 40.01 last 98.99) - open/close 4361 (time 1768252569.04 total 50.01 last 83.95) - open/close 5266 (time 1768252579.04 total 60.01 last 90.49) - open/close 6160 (time 1768252589.05 total 70.02 last 89.35) - open/close 6920 (time 1768252599.05 total 80.02 last 75.97) - open/close 7578 (time 1768252609.07 total 90.04 last 65.67) - open/close 8480 (time 1768252619.07 total 100.04 last 90.20) - open/close 9262 (time 1768252629.08 total 110.05 last 78.11) total: 10000 open/close in 118.49 seconds: 84.39 ops/second - unlinked 0 (time 1768252642 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second - open/close 841 (time 1768252741.15 total 10.01 last 84.03) - open/close 1411 (time 1768252751.16 total 20.02 last 56.91) - open/close 2221 (time 1768252761.18 total 30.04 last 80.85) - open/close 2904 (time 1768252771.19 total 40.05 last 68.22) - open/close 3607 (time 1768252781.20 total 50.06 last 70.25) - open/close 4278 (time 1768252791.22 total 60.08 last 66.95) - open/close 5078 (time 1768252801.24 total 70.10 last 79.88) - open/close 5811 (time 1768252811.24 total 80.10 last 73.30) - open/close 6369 (time 1768252821.26 total 90.12 last 55.67) - open/close 7052 (time 1768252831.27 total 100.13 last 68.23) - open/close 7880 (time 1768252841.30 total 110.16 last 82.58) - open/close 8534 (time 1768252851.32 total 120.18 last 65.29) - open/close 9220 (time 1768252861.33 total 130.19 last 68.54) - open/close 9829 (time 1768252871.34 total 140.20 last 60.82) total: 10000 open/close in 143.10 seconds: 69.88 ops/second - unlinked 0 (time 1768252879 ; total 0 ; last 0) total: 10000 unlinks in 86 seconds: 116.279068 unlinks/second - open/close 702 (time 1768252980.79 total 10.00 last 70.18) - open/close 1371 (time 1768252990.80 total 20.01 last 66.84) - open/close 1946 (time 1768253000.80 total 30.01 last 57.49) - open/close 2613 (time 1768253010.81 total 40.02 last 66.67) - open/close 3107 (time 1768253020.82 total 50.03 last 49.36) - open/close 3752 (time 1768253030.83 total 60.04 last 64.44) - open/close 4373 (time 1768253040.84 total 70.05 last 62.03) - open/close 5054 (time 1768253050.85 total 80.06 last 68.02) - open/close 5803 (time 1768253060.86 total 90.07 last 74.81) - open/close 6484 (time 1768253070.87 total 100.08 last 68.05) - open/close 7182 (time 1768253080.89 total 110.10 last 69.63) - open/close 7832 (time 1768253090.90 total 120.11 last 64.98) - open/close 8427 (time 1768253100.90 total 130.11 last 59.48) - open/close 9063 (time 1768253110.90 total 140.11 last 63.57) - open/close 9853 (time 1768253120.92 total 150.13 last 78.91) total: 10000 open/close in 152.20 seconds: 65.70 ops/second - unlinked 0 (time 1768253127 ; total 0 ; last 0) total: 10000 unlinks in 96 seconds: 104.166664 unlinks/second - open/close 651 (time 1768253239.42 total 10.01 last 65.04) - open/close 1280 (time 1768253249.43 total 20.02 last 62.85) - open/close 1997 (time 1768253259.44 total 30.03 last 71.64) - open/close 2623 (time 1768253269.45 total 40.04 last 62.51) - open/close 3419 (time 1768253279.46 total 50.05 last 79.52) - open/close 4091 (time 1768253289.46 total 60.05 last 67.19) - open/close 4741 (time 1768253299.47 total 70.06 last 64.96) - open/close 5469 (time 1768253309.47 total 80.06 last 72.80) - open/close 6251 (time 1768253319.47 total 90.07 last 78.14) - open/close 6980 (time 1768253329.48 total 100.07 last 72.84) - open/close 7720 (time 1768253339.48 total 110.07 last 74.00) - open/close 8485 (time 1768253349.49 total 120.08 last 76.42) - open/close 9317 (time 1768253359.50 total 130.09 last 83.11) - open/close 9973 (time 1768253369.52 total 140.12 last 65.45) total: 10000 open/close in 140.58 seconds: 71.13 ops/second - unlinked 0 (time 1768253374 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second - open/close 684 (time 1768253471.56 total 10.01 last 68.30) - open/close 1300 (time 1768253481.57 total 20.03 last 61.52) - open/close 2131 (time 1768253491.59 total 30.04 last 83.00) - open/close 2803 (time 1768253501.63 total 40.08 last 66.92) - open/close 3499 (time 1768253511.66 total 50.11 last 69.39) - open/close 4159 (time 1768253521.67 total 60.13 last 65.90) - open/close 4920 (time 1768253531.68 total 70.13 last 76.07) - open/close 5769 (time 1768253541.68 total 80.13 last 84.87) - open/close 6510 (time 1768253551.69 total 90.15 last 74.00) - open/close 7203 (time 1768253561.69 total 100.15 last 69.30) - open/close 7808 (time 1768253571.70 total 110.15 last 60.46) - open/close 8479 (time 1768253581.71 total 120.16 last 67.03) - open/close 9194 (time 1768253591.73 total 130.18 last 71.37) - open/close 9947 (time 1768253601.75 total 140.20 last 75.14) total: 10000 open/close in 141.05 seconds: 70.90 ops/second - unlinked 0 (time 1768253607 ; total 0 ; last 0) total: 10000 unlinks in 68 seconds: 147.058823 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg435-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.204.135@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.135@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg435-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg435-server: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg435-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg435-server: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg435-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre On OST0, 20519 used inodes rc=0 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg435-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.204.135@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.135@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: autodegrade=on sys.timeout=20 mgsnode=192.168.204.135@tcp Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: autodegrade=on sys.timeout=20 mgsnode=192.168.204.135@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:autodegrade=on lustre:sys.timeout=20 lustre:mgsnode=192.168.204.135@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg435-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg435-server: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg435-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg435-server: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg435-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock 192.168.204.135@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg435-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg435-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing unload_modules_local oleg435-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg435-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 PASS 69 (2525s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 16:37:08 (1768253828) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (3s) Stopping clients: oleg435-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg435-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 oleg435-server: oleg435-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg435-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg435-server: detected 4 online CPUs by sysfs oleg435-server: MODOPTS_LIBCFS= oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg435-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3136 sec ========== 16:38:18 (1768253898) === conf-sanity: start cleanup 16:38:20 (1768253900) === === conf-sanity: finish cleanup 16:38:23 (1768253903) ===