-----============= acceptance-small: conf-sanity ============----- Wed Jul 8 21:58:45 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg203-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex - see CLIENT_VERSION > v2_15_63-53-g1a6ef725c2 (34682473 > 34553653) for LU-16938, go 81 excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg203-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg203-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg203-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg203-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg203-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg203-server oleg203-server: oleg203-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg203-server' oleg203-server: oleg203-server.virtnet: executing load_modules_local oleg203-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg203-server: Loading modules from /home/green/git/lustre-release/lustre oleg203-server: detected 4 online CPUs by sysfs oleg203-server: MODOPTS_LIBCFS= oleg203-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg203-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg203-server: oleg203-server.virtnet: executing set_default_debug -1 all pdsh@oleg203-client: oleg203-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg203-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg203-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg203-server: oleg203-server.virtnet: executing set_default_debug -1 all pdsh@oleg203-client: oleg203-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg203-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg203-server: oleg203-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg203-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg203-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg203-server stop mds service on oleg203-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg203-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 22:00:35 (1783562435) start mds service on oleg203-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg203-server: oleg203-server.virtnet: executing set_default_debug -1 all pdsh@oleg203-client: oleg203-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg203-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg203-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg203-server: oleg203-server.virtnet: executing set_default_debug -1 all pdsh@oleg203-client: oleg203-server: ssh exited with exit code 1 Started lustre-OST0000 oleg203-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg203-client.virtnet: -o user_xattr,flock 192.168.202.103@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 92553 329 92224 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 12329 425 11904 4% /mnt/lustre[OST:0] filesystem_summary: 12233 329 11904 3% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close/unlink 577 (time 1783562470.48 total 10.01 last 57.66) - open/close/unlink 1142 (time 1783562480.49 total 20.01 last 56.48) - open/close/unlink 1809 (time 1783562490.50 total 30.02 last 66.64) - open/close/unlink 2468 (time 1783562500.51 total 40.03 last 65.84) - open/close/unlink 3072 (time 1783562510.52 total 50.05 last 60.30) - open/close/unlink 3645 (time 1783562520.53 total 60.05 last 57.29) - open/close/unlink 4302 (time 1783562530.54 total 70.06 last 65.63) - open/close/unlink 4926 (time 1783562540.56 total 80.08 last 62.27) - open/close/unlink 5505 (time 1783562550.58 total 90.10 last 57.77) - open/close/unlink 6058 (time 1783562560.59 total 100.11 last 55.23) - open/close/unlink 6578 (time 1783562570.61 total 110.14 last 51.88) - open/close/unlink 7062 (time 1783562580.63 total 120.15 last 48.33) - open/close/unlink 7595 (time 1783562590.63 total 130.16 last 53.27) - open/close/unlink 8212 (time 1783562600.64 total 140.16 last 61.68) - open/close/unlink 8840 (time 1783562610.65 total 150.17 last 62.73) - open/close/unlink 9462 (time 1783562620.65 total 160.17 last 62.19) - open/close/unlink 10000 (time 1783562628.25 total 167.77 last 70.84) - open/close/unlink 10679 (time 1783562638.26 total 177.78 last 67.83) - open/close/unlink 11346 (time 1783562648.26 total 187.79 last 66.65) - open/close/unlink 11937 (time 1783562658.27 total 197.79 last 59.08) - open/close/unlink 12503 (time 1783562668.28 total 207.80 last 56.52) - open/close/unlink 13141 (time 1783562678.28 total 217.80 last 63.78) - open/close/unlink 13869 (time 1783562688.29 total 227.81 last 72.77) - open/close/unlink 14715 (time 1783562698.29 total 237.81 last 84.56) - open/close/unlink 15461 (time 1783562708.30 total 247.82 last 74.53) - open/close/unlink 16157 (time 1783562718.31 total 257.83 last 69.54) - open/close/unlink 16946 (time 1783562728.33 total 267.85 last 78.76) - open/close/unlink 17627 (time 1783562738.34 total 277.86 last 68.02) - open/close/unlink 18360 (time 1783562748.35 total 287.87 last 73.22) - open/close/unlink 19147 (time 1783562758.35 total 297.87 last 78.68) - open/close/unlink 19835 (time 1783562768.36 total 307.88 last 68.77) - open/close/unlink 20000 (time 1783562770.58 total 310.10 last 74.35) - open/close/unlink 20761 (time 1783562780.58 total 320.10 last 76.10) - open/close/unlink 21435 (time 1783562790.58 total 330.10 last 67.39) - open/close/unlink 22060 (time 1783562800.59 total 340.11 last 62.44) - open/close/unlink 22638 (time 1783562810.60 total 350.12 last 57.75) - open/close/unlink 23164 (time 1783562820.60 total 360.12 last 52.58) - open/close/unlink 23814 (time 1783562830.60 total 370.13 last 64.97) - open/close/unlink 24373 (time 1783562840.61 total 380.13 last 55.88) - open/close/unlink 24975 (time 1783562850.62 total 390.14 last 60.14) - open/close/unlink 25573 (time 1783562860.63 total 400.15 last 59.72) - open/close/unlink 26225 (time 1783562870.65 total 410.17 last 65.08) - open/close/unlink 26818 (time 1783562880.65 total 420.17 last 59.28) - open/close/unlink 27435 (time 1783562890.68 total 430.20 last 61.51) - open/close/unlink 28108 (time 1783562900.69 total 440.21 last 67.25) - open/close/unlink 28864 (time 1783562910.70 total 450.22 last 75.54) - open/close/unlink 29607 (time 1783562920.70 total 460.22 last 74.27) - open/close/unlink 30000 (time 1783562925.23 total 464.75 last 86.81) - open/close/unlink 30879 (time 1783562935.23 total 474.75 last 87.89) - open/close/unlink 31752 (time 1783562945.23 total 484.75 last 87.27) - open/close/unlink 32487 (time 1783562955.24 total 494.76 last 73.43) - open/close/unlink 33222 (time 1783562965.25 total 504.77 last 73.48) - open/close/unlink 34015 (time 1783562975.25 total 514.78 last 79.23) - open/close/unlink 34730 (time 1783562985.26 total 524.78 last 71.45) - open/close/unlink 35573 (time 1783562995.27 total 534.79 last 84.26) - open/close/unlink 36356 (time 1783563005.27 total 544.79 last 78.27) - open/close/unlink 37251 (time 1783563015.27 total 554.80 last 89.46) - open/close/unli - open/close/unlink 584 (time 1783562470.69 total 10.02 last 58.30) - open/close/unlink 1149 (time 1783562480.69 total 20.02 last 56.50) - open/close/unlink 1809 (time 1783562490.70 total 30.02 last 65.98) - open/close/unlink 2453 (time 1783562500.71 total 40.04 last 64.31) - open/close/unlink 3057 (time 1783562510.72 total 50.04 last 60.34) - open/close/unlink 3628 (time 1783562520.73 total 60.05 last 57.04) - open/close/unlink 4283 (time 1783562530.74 total 70.07 last 65.42) - open/close/unlink 4909 (time 1783562540.75 total 80.07 last 62.58) - open/close/unlink 5487 (time 1783562550.76 total 90.09 last 57.71) - open/close/unlink 6039 (time 1783562560.76 total 100.09 last 55.20) - open/close/unlink 6555 (time 1783562570.77 total 110.09 last 51.57) - open/close/unlink 7029 (time 1783562580.78 total 120.10 last 47.35) - open/close/unlink 7565 (time 1783562590.79 total 130.11 last 53.56) - open/close/unlink 8193 (time 1783562600.80 total 140.12 last 62.73) - open/close/unlink 8810 (time 1783562610.80 total 150.12 last 61.69) - open/close/unlink 9436 (time 1783562620.81 total 160.13 last 62.55) - open/close/unlink 10000 (time 1783562628.90 total 168.22 last 69.73) - open/close/unlink 10689 (time 1783562638.90 total 178.23 last 68.85) - open/close/unlink 11349 (time 1783562648.91 total 188.24 last 65.93) - open/close/unlink 11931 (time 1783562658.93 total 198.25 last 58.13) - open/close/unlink 12498 (time 1783562668.93 total 208.26 last 56.66) - open/close/unlink 13160 (time 1783562678.93 total 218.26 last 66.19) - open/close/unlink 13881 (time 1783562688.94 total 228.26 last 72.08) - open/close/unlink 14728 (time 1783562698.94 total 238.27 last 84.64) - open/close/unlink 15475 (time 1783562708.95 total 248.28 last 74.63) - open/close/unlink 16173 (time 1783562718.95 total 258.28 last 69.79) - open/close/unlink 16969 (time 1783562728.96 total 268.29 last 79.54) - open/close/unlink 17644 (time 1783562738.97 total 278.29 last 67.48) - open/close/unlink 18380 (time 1783562748.97 total 288.30 last 73.55) - open/close/unlink 19143 (time 1783562758.98 total 298.31 last 76.24) - open/close/unlink 19838 (time 1783562768.98 total 308.31 last 69.49) - open/close/unlink 20000 (time 1783562771.08 total 310.40 last 77.26) - open/close/unlink 20758 (time 1783562781.08 total 320.40 last 75.80) - open/close/unlink 21446 (time 1783562791.09 total 330.41 last 68.76) - open/close/unlink 22048 (time 1783562801.10 total 340.43 last 60.11) - open/close/unlink 22632 (time 1783562811.11 total 350.43 last 58.36) - open/close/unlink 23160 (time 1783562821.13 total 360.46 last 52.66) - open/close/unlink 23837 (time 1783562831.16 total 370.48 last 67.53) - open/close/unlink 24402 (time 1783562841.16 total 380.49 last 56.48) - open/close/unlink 24982 (time 1783562851.16 total 390.49 last 58.00) - open/close/unlink 25590 (time 1783562861.17 total 400.50 last 60.75) - open/close/unlink 26231 (time 1783562871.17 total 410.50 last 64.09) - open/close/unlink 26828 (time 1783562881.21 total 420.53 last 59.51) - open/close/unlink 27442 (time 1783562891.21 total 430.53 last 61.38) - open/close/unlink 28132 (time 1783562901.21 total 440.54 last 68.99) - open/close/unlink 28893 (time 1783562911.22 total 450.54 last 76.06) - open/close/unlink 29638 (time 1783562921.22 total 460.54 last 74.50) - open/close/unlink 30000 (time 1783562925.41 total 464.74 last 86.27) - open/close/unlink 30882 (time 1783562935.42 total 474.75 last 88.13) - open/close/unlink 31758 (time 1783562945.43 total 484.75 last 87.56) - open/close/unlink 32487 (time 1783562955.43 total 494.75 last 72.89) - open/close/unlink 33236 (time 1783562965.43 total 504.76 last 74.85) - open/close/unlink 34024 (time 1783562975.44 total 514.77 last 78.73) - open/close/unlink 34728 (time 1783562985.46 total 524.78 last 70.29) - open/close/unlink 35572 (time 1783562995.47 total 534.79 last 84.35) - open/close/unlink 36363 (time 1783563005.47 total 544.80 last 79.06) - open/close/unlink 37253 (time 1783563015.49 total 554.81 last 88.86) - open/close/unli - open/close/unlink 575 (time 1783562470.70 total 10.01 last 57.45) - open/close/unlink 1144 (time 1783562480.72 total 20.03 last 56.81) - open/close/unlink 1804 (time 1783562490.72 total 30.03 last 65.94) - open/close/unlink 2453 (time 1783562500.73 total 40.04 last 64.87) - open/close/unlink 3054 (time 1783562510.75 total 50.05 last 60.00) - open/close/unlink 3629 (time 1783562520.76 total 60.07 last 57.44) - open/close/unlink 4285 (time 1783562530.76 total 70.07 last 65.56) - open/close/unlink 4906 (time 1783562540.76 total 80.07 last 62.09) - open/close/unlink 5484 (time 1783562550.78 total 90.09 last 57.71) - open/close/unlink 6035 (time 1783562560.79 total 100.10 last 55.02) - open/close/unlink 6555 (time 1783562570.80 total 110.11 last 51.98) - open/close/unlink 7038 (time 1783562580.80 total 120.11 last 48.27) - open/close/unlink 7563 (time 1783562590.81 total 130.12 last 52.45) - open/close/unlink 8184 (time 1783562600.82 total 140.13 last 62.04) - open/close/unlink 8800 (time 1783562610.82 total 150.13 last 61.60) - open/close/unlink 9430 (time 1783562620.83 total 160.14 last 62.99) - open/close/unlink 10000 (time 1783562628.92 total 168.23 last 70.39) - open/close/unlink 10694 (time 1783562638.93 total 178.24 last 69.33) - open/close/unlink 11344 (time 1783562648.95 total 188.26 last 64.92) - open/close/unlink 11921 (time 1783562658.96 total 198.27 last 57.64) - open/close/unlink 12491 (time 1783562668.96 total 208.27 last 56.99) - open/close/unlink 13148 (time 1783562678.97 total 218.27 last 65.65) - open/close/unlink 13874 (time 1783562688.97 total 228.28 last 72.55) - open/close/unlink 14730 (time 1783562698.97 total 238.28 last 85.59) - open/close/unlink 15475 (time 1783562708.98 total 248.29 last 74.43) - open/close/unlink 16171 (time 1783562718.98 total 258.29 last 69.58) - open/close/unlink 16957 (time 1783562728.99 total 268.30 last 78.54) - open/close/unlink 17626 (time 1783562739.00 total 278.31 last 66.85) - open/close/unlink 18355 (time 1783562749.01 total 288.32 last 72.81) - open/close/unlink 19113 (time 1783562759.03 total 298.34 last 75.69) - open/close/unlink 19813 (time 1783562769.03 total 308.34 last 69.96) - open/close/unlink 20000 (time 1783562771.44 total 310.75 last 77.64) - open/close/unlink 20754 (time 1783562781.44 total 320.75 last 75.39) - open/close/unlink 21444 (time 1783562791.45 total 330.76 last 68.93) - open/close/unlink 22035 (time 1783562801.46 total 340.77 last 59.06) - open/close/unlink 22634 (time 1783562811.46 total 350.77 last 59.89) - open/close/unlink 23144 (time 1783562821.47 total 360.78 last 50.95) - open/close/unlink 23818 (time 1783562831.47 total 370.78 last 67.38) - open/close/unlink 24381 (time 1783562841.49 total 380.80 last 56.22) - open/close/unlink 24961 (time 1783562851.49 total 390.80 last 57.96) - open/close/unlink 25574 (time 1783562861.50 total 400.81 last 61.26) - open/close/unlink 26219 (time 1783562871.51 total 410.82 last 64.44) - open/close/unlink 26820 (time 1783562881.52 total 420.83 last 60.00) - open/close/unlink 27434 (time 1783562891.54 total 430.85 last 61.33) - open/close/unlink 28123 (time 1783562901.54 total 440.85 last 68.88) - open/close/unlink 28900 (time 1783562911.55 total 450.86 last 77.64) - open/close/unlink 29638 (time 1783562921.56 total 460.87 last 73.75) - open/close/unlink 30000 (time 1783562925.63 total 464.94 last 88.86) - open/close/unlink 30876 (time 1783562935.64 total 474.95 last 87.50) - open/close/unlink 31748 (time 1783562945.65 total 484.96 last 87.10) - open/close/unlink 32473 (time 1783562955.67 total 494.98 last 72.40) - open/close/unlink 33224 (time 1783562965.67 total 504.98 last 75.07) - open/close/unlink 33995 (time 1783562975.68 total 514.99 last 77.00) - open/close/unlink 34695 (time 1783562985.70 total 525.01 last 69.89) - open/close/unlink 35543 (time 1783562995.71 total 535.02 last 84.68) - open/close/unlink 36350 (time 1783563005.71 total 545.02 last 80.69) - open/close/unlink 37230 (time 1783563015.72 total 555.03 last 87.94) - open/close/unli - open/close/unlink 592 (time 1783562470.48 total 10.00 last 59.18) - open/close/unlink 1154 (time 1783562480.49 total 20.01 last 56.17) - open/close/unlink 1810 (time 1783562490.49 total 30.01 last 65.59) - open/close/unlink 2462 (time 1783562500.50 total 40.02 last 65.15) - open/close/unlink 3059 (time 1783562510.51 total 50.03 last 59.62) - open/close/unlink 3631 (time 1783562520.53 total 60.05 last 57.10) - open/close/unlink 4289 (time 1783562530.55 total 70.07 last 65.68) - open/close/unlink 4925 (time 1783562540.56 total 80.08 last 63.53) - open/close/unlink 5498 (time 1783562550.57 total 90.09 last 57.23) - open/close/unlink 6050 (time 1783562560.58 total 100.10 last 55.13) - open/close/unlink 6581 (time 1783562570.59 total 110.11 last 53.09) - open/close/unlink 7058 (time 1783562580.60 total 120.11 last 47.66) - open/close/unlink 7592 (time 1783562590.60 total 130.12 last 53.35) - open/close/unlink 8217 (time 1783562600.61 total 140.13 last 62.48) - open/close/unlink 8850 (time 1783562610.62 total 150.14 last 63.25) - open/close/unlink 9470 (time 1783562620.62 total 160.14 last 61.99) - open/close/unlink 10000 (time 1783562628.01 total 167.53 last 71.73) - open/close/unlink 10684 (time 1783562638.02 total 177.54 last 68.28) - open/close/unlink 11349 (time 1783562648.03 total 187.55 last 66.47) - open/close/unlink 11943 (time 1783562658.03 total 197.55 last 59.38) - open/close/unlink 12513 (time 1783562668.04 total 207.56 last 56.98) - open/close/unlink 13150 (time 1783562678.05 total 217.57 last 63.61) - open/close/unlink 13883 (time 1783562688.05 total 227.57 last 73.27) - open/close/unlink 14739 (time 1783562698.06 total 237.58 last 85.53) - open/close/unlink 15489 (time 1783562708.07 total 247.59 last 74.94) - open/close/unlink 16177 (time 1783562718.07 total 257.59 last 68.80) - open/close/unlink 16971 (time 1783562728.07 total 267.59 last 79.39) - open/close/unlink 17644 (time 1783562738.08 total 277.60 last 67.23) - open/close/unlink 18382 (time 1783562748.08 total 287.60 last 73.79) - open/close/unlink 19175 (time 1783562758.09 total 297.61 last 79.22) - open/close/unlink 19864 (time 1783562768.09 total 307.61 last 68.89) - open/close/unlink 20631 (time 1783562778.10 total 317.62 last 76.65) - open/close/unlink 21274 (time 1783562788.11 total 327.63 last 64.25) - open/close/unlink 21949 (time 1783562798.11 total 337.63 last 67.44) - open/close/unlink 22540 (time 1783562808.14 total 347.66 last 58.94) - open/close/unlink 23112 (time 1783562818.15 total 357.67 last 57.16) - open/close/unlink 23683 (time 1783562828.16 total 367.68 last 57.07) - open/close/unlink 24293 (time 1783562838.16 total 377.68 last 60.99) - open/close/unlink 24896 (time 1783562848.16 total 387.68 last 60.27) - open/close/unlink 25482 (time 1783562858.16 total 397.68 last 58.58) - open/close/unlink 26105 (time 1783562868.17 total 407.69 last 62.27) - open/close/unlink 26750 (time 1783562878.18 total 417.70 last 64.46) - open/close/unlink 27353 (time 1783562888.19 total 427.71 last 60.20) - open/close/unlink 28012 (time 1783562898.20 total 437.72 last 65.83) - open/close/unlink 28739 (time 1783562908.20 total 447.72 last 72.69) - open/close/unlink 29475 (time 1783562918.22 total 457.74 last 73.48) - open/close/unlink 30000 (time 1783562924.33 total 463.85 last 85.96) - open/close/unlink 30888 (time 1783562934.33 total 473.85 last 88.75) - open/close/unlink 31755 (time 1783562944.34 total 483.86 last 86.64) - open/close/unlink 32516 (time 1783562954.34 total 493.86 last 76.06) - open/close/unlink 33243 (time 1783562964.36 total 503.88 last 72.62) - open/close/unlink 34053 (time 1783562974.37 total 513.89 last 80.90) - open/close/unlink 34772 (time 1783562984.38 total 523.90 last 71.84) - open/close/unlink 35600 (time 1783562994.38 total 533.90 last 82.77) - open/close/unlink 36370 (time 1783563004.38 total 543.90 last 77.00) - open/close/unlink 37261 (time 1783563014.38 total 553.90 last 89.08) - open/close/unlink 38176 (time 1783563024.39 total 563.91 last 91.40) - open/close/unli - open/close/unlink 593 (time 1783562470.83 total 10.01 last 59.24) - open/close/unlink 1159 (time 1783562480.84 total 20.02 last 56.57) - open/close/unlink 1827 (time 1783562490.84 total 30.02 last 66.76) - open/close/unlink 2474 (time 1783562500.85 total 40.03 last 64.66) - open/close/unlink 3058 (time 1783562510.85 total 50.03 last 58.38) - open/close/unlink 3628 (time 1783562520.85 total 60.03 last 56.99) - open/close/unlink 4287 (time 1783562530.86 total 70.04 last 65.87) - open/close/unlink 4907 (time 1783562540.88 total 80.06 last 61.86) - open/close/unlink 5486 (time 1783562550.89 total 90.07 last 57.84) - open/close/unlink 6036 (time 1783562560.91 total 100.09 last 54.90) - open/close/unlink 6560 (time 1783562570.93 total 110.10 last 52.32) - open/close/unlink 7040 (time 1783562580.94 total 120.12 last 47.94) - open/close/unlink 7571 (time 1783562590.94 total 130.12 last 53.07) - open/close/unlink 8200 (time 1783562600.95 total 140.13 last 62.85) - open/close/unlink 8824 (time 1783562610.96 total 150.14 last 62.31) - open/close/unlink 9448 (time 1783562620.98 total 160.15 last 62.33) - open/close/unlink 10000 (time 1783562628.90 total 168.07 last 69.68) - open/close/unlink 10690 (time 1783562638.91 total 178.09 last 68.91) - open/close/unlink 11345 (time 1783562648.92 total 188.10 last 65.43) - open/close/unlink 11926 (time 1783562658.93 total 198.11 last 58.03) - open/close/unlink 12497 (time 1783562668.94 total 208.12 last 57.04) - open/close/unlink 13162 (time 1783562678.95 total 218.12 last 66.48) - open/close/unlink 13888 (time 1783562688.95 total 228.13 last 72.57) - open/close/unlink 14752 (time 1783562698.96 total 238.14 last 86.35) - open/close/unlink 15494 (time 1783562708.96 total 248.14 last 74.19) - open/close/unlink 16191 (time 1783562718.96 total 258.14 last 69.67) - open/close/unlink 16990 (time 1783562728.98 total 268.15 last 79.81) - open/close/unlink 17665 (time 1783562738.98 total 278.16 last 67.49) - open/close/unlink 18404 (time 1783562748.98 total 288.16 last 73.87) - open/close/unlink 19175 (time 1783562758.99 total 298.17 last 77.00) - open/close/unlink 19876 (time 1783562769.01 total 308.18 last 70.01) - open/close/unlink 20644 (time 1783562779.03 total 318.20 last 76.66) - open/close/unlink 21288 (time 1783562789.03 total 328.21 last 64.34) - open/close/unlink 21969 (time 1783562799.05 total 338.23 last 68.00) - open/close/unlink 22555 (time 1783562809.05 total 348.23 last 58.56) - open/close/unlink 23108 (time 1783562819.08 total 358.26 last 55.15) - open/close/unlink 23710 (time 1783562829.08 total 368.26 last 60.20) - open/close/unlink 24313 (time 1783562839.10 total 378.27 last 60.22) - open/close/unlink 24917 (time 1783562849.11 total 388.29 last 60.30) - open/close/unlink 25521 (time 1783562859.12 total 398.30 last 60.35) - open/close/unlink 26139 (time 1783562869.13 total 408.30 last 61.76) - open/close/unlink 26771 (time 1783562879.13 total 418.31 last 63.19) - open/close/unlink 27379 (time 1783562889.13 total 428.31 last 60.76) - open/close/unlink 28036 (time 1783562899.14 total 438.32 last 65.64) - open/close/unlink 28780 (time 1783562909.15 total 448.33 last 74.34) - open/close/unlink 29503 (time 1783562919.16 total 458.34 last 72.22) - open/close/unlink 30000 (time 1783562924.88 total 464.05 last 86.96) - open/close/unlink 30889 (time 1783562934.89 total 474.06 last 88.81) - open/close/unlink 31765 (time 1783562944.90 total 484.07 last 87.52) - open/close/unlink 32514 (time 1783562954.90 total 494.08 last 74.86) - open/close/unlink 33249 (time 1783562964.90 total 504.08 last 73.48) - open/close/unlink 34045 (time 1783562974.91 total 514.09 last 79.54) - open/close/unlink 34754 (time 1783562984.92 total 524.10 last 70.84) - open/close/unlink 35589 (time 1783562994.92 total 534.10 last 83.46) - open/close/unlink 36370 (time 1783563004.93 total 544.11 last 78.05) - open/close/unlink 37263 (time 1783563014.94 total 554.12 last 89.23) - open/close/unlink 38178 (time 1783563024.94 total 564.12 last 91.46) - open/close/unlink 39302 (time 1783563034.40 total 573.92 last 112.51) - open/close/unlink 40000 (time 1783563041.30 total 580.82 last 101.22) - open/close/unlink 41132 (time 1783563051.31 total 590.83 last 113.04) - open/close/unlink 42236 (time 1783563061.31 total 600.83 last 110.39) - open/close/unlink 43086 (time 1783563071.33 total 610.85 last 84.88) - open/close/unlink 43941 (time 1783563081.34 total 620.86 last 85.42) - open/close/unlink 44772 (time 1783563091.35 total 630.87 last 83.00) - open/close/unlink 45544 (time 1783563101.36 total 640.88 last 77.12) - open/close/unlink 46505 (time 1783563111.37 total 650.88 last 96.03) - open/close/unlink 47609 (time 1783563121.37 total 660.89 last 110.40) - open/close/unlink 48516 (time 1783563131.37 total 670.89 last 90.61) - open/close/unlink 49400 (time 1783563141.38 total 680.90 last 88.32) - open/close/unlink 50000 (time 1783563148.32 total 687.84 last 86.46) - open/close/unlink 50891 (time 1783563158.33 total 697.85 last 89.03) - open/close/unlink 51983 (time 1783563168.34 total 707.86 last 109.16) - open/close/unlink 53005 (time 1783563178.34 total 717.86 last 102.18) - open/close/unlink 53922 (time 1783563188.34 total 727.86 last 91.70) - open/close/unlink 54801 (time 1783563198.34 total 737.86 last 87.87) - open/close/unlink 55860 (time 1783563208.34 total 747.86 last 105.87) - open/close/unlink 56909 (time 1783563218.35 total 757.87 last 104.84) - open/close/unlink 58135 (time 1783563228.36 total 767.88 last 122.46) - open/close/unlink 59396 (time 1783563238.36 total 777.88 last 126.06) - open/close/unlink 60000 (time 1783563242.63 total 782.15 last 141.63) - open/close/unlink 61336 (time 1783563252.64 total 792.16 last 133.52) - open/close/unlink 62594 (time 1783563262.64 total 802.16 last 125.75) - open/close/unlink 63930 (time 1783563272.64 total 812.16 last 133.56) total: 64768 open/close/unlink in 818.35 seconds: 79.14 ops/second nk 39296 (time 1783563034.95 total 574.12 last 111.77) - open/close/unlink 40000 (time 1783563041.67 total 580.84 last 104.75) - open/close/unlink 41133 (time 1783563051.67 total 590.85 last 113.22) - open/close/unlink 42252 (time 1783563061.68 total 600.86 last 111.84) - open/close/unlink 43095 (time 1783563071.68 total 610.86 last 84.28) - open/close/unlink 43925 (time 1783563081.68 total 620.86 last 82.99) - open/close/unlink 44762 (time 1783563091.69 total 630.87 last 83.66) - open/close/unlink 45533 (time 1783563101.69 total 640.87 last 77.09) - open/close/unlink 46491 (time 1783563111.70 total 650.88 last 95.71) - open/close/unlink 47604 (time 1783563121.70 total 660.88 last 111.25) - open/close/unlink 48517 (time 1783563131.70 total 670.88 last 91.29) - open/close/unlink 49423 (time 1783563141.70 total 680.88 last 90.60) - open/close/unlink 50000 (time 1783563148.48 total 687.66 last 85.12) - open/close/unlink 50897 (time 1783563158.49 total 697.67 last 89.65) - open/close/unlink 51978 (time 1783563168.49 total 707.67 last 108.07) - open/close/unlink 53000 (time 1783563178.49 total 717.67 last 102.20) - open/close/unlink 53918 (time 1783563188.50 total 727.68 last 91.70) - open/close/unlink 54788 (time 1783563198.51 total 737.69 last 86.94) - open/close/unlink 55853 (time 1783563208.51 total 747.69 last 106.44) - open/close/unlink 56897 (time 1783563218.52 total 757.70 last 104.35) - open/close/unlink 58115 (time 1783563228.53 total 767.71 last 121.68) - open/close/unlink 59388 (time 1783563238.53 total 777.71 last 127.26) - open/close/unlink 60000 (time 1783563242.87 total 782.05 last 141.14) - open/close/unlink 61329 (time 1783563252.87 total 792.05 last 132.82) - open/close/unlink 62589 (time 1783563262.89 total 802.06 last 125.85) - open/close/unlink 63926 (time 1783563272.89 total 812.07 last 133.64) total: 64768 open/close/unlink in 818.72 seconds: 79.11 ops/second nk 38178 (time 1783563025.28 total 564.80 last 92.65) - open/close/unlink 39270 (time 1783563035.28 total 574.80 last 109.18) - open/close/unlink 40000 (time 1783563042.15 total 581.67 last 106.30) - open/close/unlink 41132 (time 1783563052.15 total 591.67 last 113.18) - open/close/unlink 42257 (time 1783563062.15 total 601.67 last 112.48) - open/close/unlink 43079 (time 1783563072.15 total 611.68 last 82.18) - open/close/unlink 43916 (time 1783563082.17 total 621.69 last 83.57) - open/close/unlink 44756 (time 1783563092.18 total 631.70 last 83.90) - open/close/unlink 45509 (time 1783563102.18 total 641.71 last 75.29) - open/close/unlink 46489 (time 1783563112.19 total 651.71 last 97.97) - open/close/unlink 47581 (time 1783563122.19 total 661.71 last 109.13) - open/close/unlink 48480 (time 1783563132.22 total 671.74 last 89.65) - open/close/unlink 49392 (time 1783563142.22 total 681.74 last 91.18) - open/close/unlink 50000 (time 1783563149.24 total 688.77 last 86.58) - open/close/unlink 50900 (time 1783563159.24 total 698.77 last 90.00) - open/close/unlink 51974 (time 1783563169.25 total 708.77 last 107.37) - open/close/unlink 53015 (time 1783563179.26 total 718.78 last 104.02) - open/close/unlink 53913 (time 1783563189.27 total 728.79 last 89.65) - open/close/unlink 54786 (time 1783563199.27 total 738.80 last 87.29) - open/close/unlink 55848 (time 1783563209.27 total 748.80 last 106.18) - open/close/unlink 56909 (time 1783563219.28 total 758.80 last 106.04) - open/close/unlink 58130 (time 1783563229.28 total 768.80 last 122.09) - open/close/unlink 59442 (time 1783563239.28 total 778.80 last 131.19) - open/close/unlink 60000 (time 1783563243.27 total 782.79 last 140.03) - open/close/unlink 61324 (time 1783563253.27 total 792.79 last 132.34) - open/close/unlink 62570 (time 1783563263.27 total 802.80 last 124.58) - open/close/unlink 63894 (time 1783563273.28 total 812.80 last 132.32) total: 64768 open/close/unlink in 821.91 seconds: 78.80 ops/second nk 38203 (time 1783563025.49 total 564.82 last 94.95) - open/close/unlink 39292 (time 1783563035.49 total 574.82 last 108.88) - open/close/unlink 40000 (time 1783563042.09 total 581.41 last 107.38) - open/close/unlink 41121 (time 1783563052.09 total 591.42 last 112.05) - open/close/unlink 42236 (time 1783563062.10 total 601.42 last 111.40) - open/close/unlink 43068 (time 1783563072.11 total 611.43 last 83.16) - open/close/unlink 43901 (time 1783563082.12 total 621.44 last 83.21) - open/close/unlink 44738 (time 1783563092.12 total 631.44 last 83.67) - open/close/unlink 45503 (time 1783563102.13 total 641.46 last 76.39) - open/close/unlink 46475 (time 1783563112.14 total 651.47 last 97.14) - open/close/unlink 47573 (time 1783563122.15 total 661.47 last 109.73) - open/close/unlink 48471 (time 1783563132.15 total 671.47 last 89.78) - open/close/unlink 49388 (time 1783563142.15 total 681.47 last 91.69) - open/close/unlink 50000 (time 1783563149.22 total 688.54 last 86.59) - open/close/unlink 50899 (time 1783563159.22 total 698.54 last 89.89) - open/close/unlink 51964 (time 1783563169.22 total 708.55 last 106.46) - open/close/unlink 52995 (time 1783563179.23 total 718.55 last 103.03) - open/close/unlink 53901 (time 1783563189.24 total 728.57 last 90.47) - open/close/unlink 54775 (time 1783563199.25 total 738.57 last 87.36) - open/close/unlink 55829 (time 1783563209.25 total 748.58 last 105.34) - open/close/unlink 56879 (time 1783563219.26 total 758.59 last 104.91) - open/close/unlink 58104 (time 1783563229.27 total 768.60 last 122.40) - open/close/unlink 59421 (time 1783563239.27 total 778.60 last 131.67) - open/close/unlink 60000 (time 1783563243.32 total 782.64 last 143.07) - open/close/unlink 61318 (time 1783563253.33 total 792.65 last 131.73) - open/close/unlink 62568 (time 1783563263.33 total 802.66 last 124.92) - open/close/unlink 63888 (time 1783563273.34 total 812.66 last 131.91) total: 64768 open/close/unlink in 821.83 seconds: 78.81 ops/second nk 38181 (time 1783563025.72 total 565.03 last 95.10) - open/close/unlink 39251 (time 1783563035.73 total 575.04 last 106.92) - open/close/unlink 40000 (time 1783563042.54 total 581.85 last 110.00) - open/close/unlink 41120 (time 1783563052.54 total 591.85 last 111.98) - open/close/unlink 42241 (time 1783563062.54 total 601.85 last 112.05) - open/close/unlink 43059 (time 1783563072.55 total 611.86 last 81.75) - open/close/unlink 43890 (time 1783563082.55 total 621.86 last 83.06) - open/close/unlink 44722 (time 1783563092.56 total 631.87 last 83.13) - open/close/unlink 45486 (time 1783563102.57 total 641.88 last 76.33) - open/close/unlink 46473 (time 1783563112.58 total 651.89 last 98.63) - open/close/unlink 47580 (time 1783563122.59 total 661.90 last 110.63) - open/close/unlink 48470 (time 1783563132.59 total 671.90 last 88.93) - open/close/unlink 49377 (time 1783563142.60 total 681.91 last 90.63) - open/close/unlink 50000 (time 1783563149.60 total 688.91 last 88.98) - open/close/unlink 50912 (time 1783563159.61 total 698.92 last 91.14) - open/close/unlink 51977 (time 1783563169.61 total 708.92 last 106.50) - open/close/unlink 52994 (time 1783563179.61 total 718.92 last 101.66) - open/close/unlink 53901 (time 1783563189.62 total 728.93 last 90.60) - open/close/unlink 54781 (time 1783563199.63 total 738.94 last 87.97) - open/close/unlink 55863 (time 1783563209.63 total 748.94 last 108.18) - open/close/unlink 56911 (time 1783563219.63 total 758.94 last 104.76) - open/close/unlink 58119 (time 1783563229.64 total 768.95 last 120.73) - open/close/unlink 59426 (time 1783563239.65 total 778.96 last 130.57) - open/close/unlink 60000 (time 1783563243.57 total 782.88 last 146.26) - open/close/unlink 61329 (time 1783563253.57 total 792.88 last 132.89) - open/close/unlink 62590 (time 1783563263.57 total 802.88 last 126.10) - open/close/unlink 63910 (time 1783563273.58 total 812.89 last 131.94) total: 64768 open/close/unlink in 822.54 seconds: 78.74 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete oleg203-server: cannot open 'lustre-ost2': no such pool pdsh@oleg203-client: oleg203-server: ssh exited with exit code 255 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg203-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg203-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg203-server stop mds service on oleg203-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg203-server unloading modules via unload_modules_local on: 'oleg203-server' oleg203-server: oleg203-server.virtnet: executing unload_modules_local oleg203-server: modules unloaded. pdsh@oleg203-client: oleg203-client: ssh exited with exit code 2 pdsh@oleg203-client: oleg203-server: ssh exited with exit code 2 pdsh@oleg203-client: oleg203-client: ssh exited with exit code 2 pdsh@oleg203-client: oleg203-server: ssh exited with exit code 2 PASS 106 (884s) Stopping clients: oleg203-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg203-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg203-client: oleg203-server: ssh exited with exit code 2 oleg203-server: oleg203-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg203-server' oleg203-server: oleg203-server.virtnet: executing load_modules_local oleg203-server: Loading modules from /home/green/git/lustre-release/lustre oleg203-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg203-server: detected 4 online CPUs by sysfs oleg203-server: MODOPTS_LIBCFS= oleg203-server: Force libcfs to create 2 CPU partitions oleg203-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg203-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg203-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1022 sec ========== 22:15:48 (1783563348) === conf-sanity: start cleanup 22:15:49 (1783563349) === === conf-sanity: finish cleanup 22:15:51 (1783563351) ===