-----============= acceptance-small: conf-sanity ============----- Thu Mar 26 05:37:26 EDT 2026 oleg154-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg154-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg154-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg154-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg154-server oleg154-server: oleg154-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing load_modules_local oleg154-server: Loading modules from /home/green/git/lustre-release/lustre oleg154-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg154-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg154-server: detected 4 online CPUs by sysfs oleg154-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg154-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg154-server: oleg154-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg154-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 05:40:29 (1774518029) Stopping clients: oleg154-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg154-client.virtnet /mnt/lustre2 (opts:-f) oleg154-server: oleg154-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing load_modules_local oleg154-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg154-server: Loading modules from /home/green/git/lustre-release/lustre oleg154-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg154-server: detected 4 online CPUs by sysfs oleg154-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg154-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg154-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg154-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg154-server: oleg154-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg154-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg154-client.virtnet: -o user_xattr,flock oleg154-server@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 16166 358 15808 3% /mnt/lustre[OST:0] filesystem_summary: 16137 329 15808 3% /mnt/lustre fail_loc=0x1312 fail_val=5 - open/close 531 (time 1774518174.79 total 10.02 last 52.99) - open/close 912 (time 1774518184.80 total 20.03 last 38.06) - open/close 1515 (time 1774518194.82 total 30.05 last 60.21) - open/close 2243 (time 1774518204.83 total 40.06 last 72.70) - open/close 2886 (time 1774518214.84 total 50.07 last 64.24) - open/close 3575 (time 1774518224.84 total 60.07 last 68.90) - open/close 4210 (time 1774518234.86 total 70.08 last 63.41) - open/close 5082 (time 1774518244.86 total 80.09 last 87.14) - open/close 5804 (time 1774518254.87 total 90.09 last 72.16) - open/close 6339 (time 1774518264.88 total 100.11 last 53.44) - open/close 6832 (time 1774518274.88 total 110.11 last 49.30) - open/close 7448 (time 1774518284.90 total 120.13 last 61.45) - open/close 7952 (time 1774518294.92 total 130.15 last 50.30) - open/close 8494 (time 1774518304.93 total 140.15 last 54.17) - open/close 9202 (time 1774518314.95 total 150.17 last 70.67) - open/close 9846 (time 1774518324.95 total 160.18 last 64.36) - open/close 10000 (time 1774518327.26 total 162.49 last 66.63) - open/close 10730 (time 1774518337.27 total 172.50 last 72.94) - open/close 11421 (time 1774518347.29 total 182.51 last 68.99) - open/close 12011 (time 1774518357.32 total 192.55 last 58.79) - open/close 12702 (time 1774518367.33 total 202.56 last 69.05) - open/close 13351 (time 1774518377.34 total 212.57 last 64.83) - open/close 13977 (time 1774518387.35 total 222.58 last 62.53) - open/close 14691 (time 1774518397.38 total 232.60 last 71.24) - open/close 15314 (time 1774518407.38 total 242.60 last 62.29) - open/close 15902 (time 1774518417.38 total 252.61 last 58.78) - open/close 16509 (time 1774518427.39 total 262.62 last 60.64) - open/close 17132 (time 1774518437.41 total 272.63 last 62.19) - open/close 17612 (time 1774518447.41 total 282.64 last 47.99) - open/close 18109 (time 1774518457.42 total 292.65 last 49.65) - open/close 18761 (time 1774518467.43 total 302.65 last 65.15) - open/close 19528 (time 1774518477.43 total 312.66 last 76.69) - open/close 20000 (time 1774518484.15 total 319.38 last 70.20) - open/close 20635 (time 1774518494.16 total 329.39 last 63.45) - open/close 21296 (time 1774518504.16 total 339.39 last 66.08) - open/close 22074 (time 1774518514.19 total 349.41 last 77.62) - open/close 22806 (time 1774518524.21 total 359.43 last 73.06) - open/close 23568 (time 1774518534.22 total 369.45 last 76.08) - open/close 24313 (time 1774518544.23 total 379.46 last 74.40) - open/close 25089 (time 1774518554.24 total 389.46 last 77.57) - open/close 25807 (time 1774518564.27 total 399.49 last 71.60) - open/close 26527 (time 1774518574.28 total 409.51 last 71.89) - open/close 27213 (time 1774518584.29 total 419.52 last 68.54) - open/close 27944 (time 1774518594.29 total 429.52 last 73.07) - open/close 28670 (time 1774518604.30 total 439.53 last 72.56) - open/close 29520 (time 1774518614.30 total 449.53 last 84.96) - open/close 30000 (time 1774518619.32 total 454.55 last 95.65) - open/close 30764 (time 1774518629.33 total 464.56 last 76.34) - open/close 31525 (time 1774518639.34 total 474.57 last 76.01) - open/close 32368 (time 1774518649.34 total 484.57 last 84.30) - open/close 33283 (time 1774518659.35 total 494.57 last 91.47) - open/close 34205 (time 1774518669.35 total 504.58 last 92.15) - open/close 35078 (time 1774518679.35 total 514.58 last 87.27) - open/close 36002 (time 1774518689.36 total 524.59 last 92.34) - open/close 36948 (time 1774518699.36 total 534.59 last 94.57) - open/close 37912 (time 1774518709.37 total 544.60 last 96.34) - open/close 38847 (time 1774518719.37 total 554.60 last 93.50) - open/close 39904 (time 1774518729.37 total 564.60 last 105.66) - open/close 40873 (time 1774518739.38 total 574.61 last 96.82) - open/close 41778 (time 1774518749.39 total 584.61 last 90.45) - open/close 42940 (time 1774518759.40 total 594.62 last 116.09) - open/close 44030 (time 1774518769.40 total 604.63 last 108.99) - open/close 44987 (time 1774518779.40 total 614.63 last 95.65) - open/close 45947 (time 1774518789.42 total 624.65 last 95.85) - open/close 46900 (time 1774518799.42 total 634.65 last 95.29) - open/close 47845 (time 1774518809.43 total 644.65 last 94.44) - open/close 48816 (time 1774518819.43 total 654.66 last 97.04) - open/close 49819 (time 1774518829.44 total 664.67 last 100.21) - open/close 50822 (time 1774518839.45 total 674.67 last 100.27) - open/close 51974 (time 1774518849.45 total 684.68 last 115.11) - open/close 52998 (time 1774518859.45 total 694.68 last 102.40) - open/close 53908 (time 1774518869.46 total 704.69 last 90.94) - open/close 54872 (time 1774518879.46 total 714.69 last 96.39) - open/close 55948 (time 1774518889.46 total 724.69 last 107.59) - open/close 57068 (time 1774518899.47 total 734.69 last 111.98) - open/close 58263 (time 1774518909.47 total 744.70 last 119.46) - open/close 59265 (time 1774518919.47 total 754.70 last 100.15) - open/close 60000 (time 1774518927.07 total 762.29 last 96.80) - open/close 61084 (time 1774518937.07 total 772.30 last 108.38) - open/close 62099 (time 1774518947.07 total 782.30 last 101.49) - open/close 63194 (time 1774518957.08 total 792.31 last 109.36) - open/close 64268 (time 1774518967.09 total 802.32 last 107.29) total: 64768 open/close in 807.04 seconds: 80.25 ops/second - /unlink 1799 (time 1774518984.41 total 10.00 last 179.87) - /unlink 3594 (time 1774518994.41 total 20.01 last 179.42) - /unlink 5250 (time 1774519004.41 total 30.01 last 165.57) - /unlink 7245 (time 1774519014.42 total 40.01 last 199.43) - /unlink 9287 (time 1774519024.42 total 50.02 last 204.12) - /unlink 10000 (time 1774519027.98 total 53.57 last 200.35) - /unlink 12017 (time 1774519037.98 total 63.58 last 201.68) - /unlink 13926 (time 1774519047.99 total 73.58 last 190.82) - /unlink 15761 (time 1774519057.99 total 83.58 last 183.47) - /unlink 17639 (time 1774519067.99 total 93.59 last 187.69) - /unlink 19826 (time 1774519077.99 total 103.59 last 218.70) - /unlink 22047 (time 1774519088.00 total 113.59 last 222.06) - /unlink 24808 (time 1774519098.00 total 123.59 last 275.99) - /unlink 27035 (time 1774519108.01 total 133.60 last 222.57) - /unlink 29469 (time 1774519118.01 total 143.60 last 243.27) - /unlink 30000 (time 1774519120.49 total 146.08 last 214.19) - /unlink 32244 (time 1774519130.49 total 156.08 last 224.37) - /unlink 34722 (time 1774519140.49 total 166.09 last 247.72) - /unlink 37238 (time 1774519150.50 total 176.09 last 251.56) - /unlink 39235 (time 1774519160.50 total 186.09 last 199.65) - /unlink 40000 (time 1774519165.03 total 190.63 last 168.73) - /unlink 41948 (time 1774519175.04 total 200.63 last 194.72) - /unlink 43970 (time 1774519185.04 total 210.63 last 202.17) - /unlink 46218 (time 1774519195.04 total 220.63 last 224.78) - /unlink 48397 (time 1774519205.04 total 230.63 last 217.86) - /unlink 50000 (time 1774519211.84 total 237.43 last 235.82) - /unlink 52235 (time 1774519221.84 total 247.43 last 223.45) - /unlink 54641 (time 1774519231.84 total 257.44 last 240.54) - /unlink 57269 (time 1774519241.84 total 267.44 last 262.78) - /unlink 59767 (time 1774519251.85 total 277.44 last 249.71) - /unlink 62186 (time 1774519261.85 total 287.44 last 241.85) - /unlink 64567 (time 1774519271.85 total 297.45 last 237.98) total: 64768 /unlink in 298.50 seconds: 216.98 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1437 (time 1774519303.79 total 10.01 last 143.59) - open/close 2802 (time 1774519313.79 total 20.01 last 136.50) - open/close 4070 (time 1774519323.79 total 30.01 last 126.77) - open/close 5154 (time 1774519333.80 total 40.01 last 108.37) - open/close 6243 (time 1774519343.80 total 50.02 last 108.86) - open/close 7507 (time 1774519353.81 total 60.02 last 126.32) - open/close 8825 (time 1774519363.81 total 70.03 last 131.78) - open/close 10000 (time 1774519373.21 total 79.42 last 125.02) - open/close 11290 (time 1774519383.21 total 89.43 last 128.95) - open/close 12526 (time 1774519393.22 total 99.43 last 123.53) - open/close 13916 (time 1774519403.22 total 109.44 last 138.95) - open/close 15256 (time 1774519413.22 total 119.44 last 134.00) - open/close 16803 (time 1774519423.22 total 129.44 last 154.69) - open/close 18312 (time 1774519433.23 total 139.44 last 150.84) - open/close 19731 (time 1774519443.23 total 149.44 last 141.89) - open/close 20000 (time 1774519445.28 total 151.50 last 130.76) - open/close 21319 (time 1774519455.29 total 161.50 last 131.88) - open/close 22897 (time 1774519465.29 total 171.51 last 157.74) - open/close 24337 (time 1774519475.30 total 181.52 last 143.78) - open/close 25639 (time 1774519485.31 total 191.52 last 130.19) - open/close 27030 (time 1774519495.31 total 201.52 last 139.10) - open/close 28661 (time 1774519505.31 total 211.52 last 163.06) - open/close 30000 (time 1774519514.21 total 220.43 last 150.41) - open/close 31563 (time 1774519524.21 total 230.43 last 156.29) - open/close 33147 (time 1774519534.21 total 240.43 last 158.37) - open/close 34642 (time 1774519544.22 total 250.44 last 149.36) - open/close 35865 (time 1774519554.22 total 260.44 last 122.30) - open/close 37113 (time 1774519564.23 total 270.45 last 124.72) - open/close 38364 (time 1774519574.24 total 280.45 last 125.02) - open/close 39648 (time 1774519584.24 total 290.46 last 128.35) - open/close 40000 (time 1774519586.94 total 293.15 last 130.46) - open/close 41274 (time 1774519596.95 total 303.16 last 127.28) - open/close 42532 (time 1774519606.95 total 313.17 last 125.76) - open/close 43906 (time 1774519616.95 total 323.17 last 137.38) - open/close 45614 (time 1774519626.95 total 333.17 last 170.76) - open/close 46874 (time 1774519636.95 total 343.17 last 126.00) - open/close 48257 (time 1774519646.96 total 353.17 last 138.28) - open/close 49997 (time 1774519656.96 total 363.17 last 173.96) - open/close 51827 (time 1774519666.96 total 373.17 last 183.00) - open/close 53322 (time 1774519676.96 total 383.18 last 149.44) - open/close 55358 (time 1774519686.96 total 393.18 last 203.58) - open/close 57009 (time 1774519696.97 total 403.18 last 165.04) - open/close 58598 (time 1774519706.97 total 413.18 last 158.87) - open/close 60000 (time 1774519715.04 total 421.26 last 173.66) - open/close 62312 (time 1774519725.04 total 431.26 last 231.18) - open/close 64642 (time 1774519735.04 total 441.26 last 232.98) total: 64768 open/close in 441.91 seconds: 146.56 ops/second - /unlink 3611 (time 1774519747.52 total 10.00 last 361.08) - /unlink 6568 (time 1774519757.52 total 20.00 last 295.69) - /unlink 9646 (time 1774519767.52 total 30.00 last 307.78) - /unlink 13032 (time 1774519777.52 total 40.00 last 338.57) - /unlink 16534 (time 1774519787.52 total 50.00 last 350.15) - /unlink 20000 (time 1774519795.96 total 58.44 last 410.89) - /unlink 24492 (time 1774519805.96 total 68.44 last 449.19) - /unlink 27524 (time 1774519815.96 total 78.44 last 303.13) - /unlink 30000 (time 1774519824.44 total 86.92 last 292.00) - /unlink 32918 (time 1774519834.44 total 96.92 last 291.74) - /unlink 35990 (time 1774519844.44 total 106.93 last 307.10) - /unlink 38765 (time 1774519854.45 total 116.93 last 277.45) - /unlink 40000 (time 1774519858.30 total 120.78 last 320.33) - /unlink 43702 (time 1774519868.30 total 130.79 last 370.15) - /unlink 47136 (time 1774519878.30 total 140.79 last 343.36) - /unlink 50000 (time 1774519888.30 total 150.78 last 286.48) - /unlink 53328 (time 1774519898.30 total 160.78 last 332.80) - /unlink 56534 (time 1774519908.30 total 170.79 last 320.54) - /unlink 59917 (time 1774519918.30 total 180.79 last 338.26) - /unlink 62986 (time 1774519928.30 total 190.79 last 306.89) total: 64768 /unlink in 195.95 seconds: 330.54 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1993 (time 1774519965.06 total 10.00 last 199.26) - open/close 3751 (time 1774519975.06 total 20.00 last 175.78) - open/close 5755 (time 1774519985.07 total 30.01 last 200.33) - open/close 7998 (time 1774519995.07 total 40.01 last 224.22) - open/close 9639 (time 1774520005.07 total 50.01 last 164.05) - open/close 11374 (time 1774520015.07 total 60.01 last 173.49) - open/close 12942 (time 1774520025.08 total 70.02 last 156.76) - open/close 15079 (time 1774520035.08 total 80.02 last 213.69) - open/close 17240 (time 1774520045.08 total 90.02 last 216.06) - open/close 19690 (time 1774520055.08 total 100.02 last 244.84) - open/close 22462 (time 1774520065.09 total 110.03 last 277.01) - open/close 24820 (time 1774520075.09 total 120.03 last 235.75) - open/close 27270 (time 1774520085.09 total 130.03 last 244.97) - open/close 30000 (time 1774520094.78 total 139.72 last 281.92) - open/close 33284 (time 1774520104.78 total 149.72 last 328.35) - open/close 36928 (time 1774520114.78 total 159.72 last 364.37) - open/close 40000 (time 1774520122.93 total 167.88 last 376.71) - open/close 42425 (time 1774520132.94 total 177.88 last 242.40) - open/close 44710 (time 1774520142.94 total 187.88 last 228.43) - open/close 46785 (time 1774520152.94 total 197.88 last 207.46) - open/close 48967 (time 1774520162.94 total 207.89 last 218.19) - open/close 50000 (time 1774520167.52 total 212.46 last 225.86) - open/close 52604 (time 1774520177.52 total 222.46 last 260.33) - open/close 55906 (time 1774520187.52 total 232.46 last 330.16) - open/close 59164 (time 1774520197.52 total 242.46 last 325.80) - open/close 60000 (time 1774520199.63 total 244.57 last 396.17) - open/close 63282 (time 1774520209.64 total 254.58 last 328.04) total: 64768 open/close in 259.85 seconds: 249.26 ops/second - /unlink 5610 (time 1774520226.42 total 10.00 last 560.91) - /unlink 10000 (time 1774520235.13 total 18.71 last 504.29) - /unlink 14750 (time 1774520245.13 total 28.71 last 474.91) - /unlink 19230 (time 1774520255.13 total 38.71 last 447.96) - /unlink 24192 (time 1774520265.13 total 48.71 last 496.13) - /unlink 29271 (time 1774520275.13 total 58.71 last 507.84) - /unlink 34755 (time 1774520285.14 total 68.71 last 548.31) - /unlink 40000 (time 1774520294.53 total 78.11 last 558.47) - /unlink 45247 (time 1774520304.53 total 88.11 last 524.66) - /unlink 50000 (time 1774520311.35 total 94.93 last 696.35) - /unlink 56106 (time 1774520321.35 total 104.93 last 610.56) - /unlink 60000 (time 1774520327.02 total 110.60 last 686.98) total: 64768 /unlink in 119.19 seconds: 543.40 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 4475 (time 1774520363.13 total 10.00 last 447.50) - open/close 6822 (time 1774520373.13 total 20.00 last 234.66) - open/close 9390 (time 1774520383.14 total 30.01 last 256.58) - open/close 10000 (time 1774520386.01 total 32.88 last 212.29) - open/close 12339 (time 1774520396.02 total 42.89 last 233.86) - open/close 14998 (time 1774520406.02 total 52.89 last 265.89) - open/close 17927 (time 1774520416.02 total 62.89 last 292.85) - open/close 20000 (time 1774520420.45 total 67.32 last 467.54) - open/close 24758 (time 1774520430.45 total 77.32 last 475.78) - open/close 29634 (time 1774520440.45 total 87.32 last 487.60) - open/close 34222 (time 1774520450.45 total 97.32 last 458.77) - open/close 39001 (time 1774520460.45 total 107.32 last 477.85) - open/close 40000 (time 1774520462.68 total 109.55 last 448.61) - open/close 44356 (time 1774520472.68 total 119.55 last 435.57) - open/close 48490 (time 1774520482.68 total 129.55 last 413.37) - open/close 50000 (time 1774520485.71 total 132.58 last 498.17) - open/close 54649 (time 1774520495.72 total 142.58 last 464.82) - open/close 59327 (time 1774520505.72 total 152.59 last 467.74) - open/close 63847 (time 1774520515.72 total 162.59 last 451.99) total: 64768 open/close in 164.51 seconds: 393.71 ops/second - /unlink 7165 (time 1774520528.96 total 10.00 last 716.48) - /unlink 10000 (time 1774520532.54 total 13.57 last 793.22) - /unlink 17269 (time 1774520542.54 total 23.57 last 726.86) - /unlink 20000 (time 1774520546.28 total 27.32 last 729.50) - /unlink 27938 (time 1774520556.28 total 37.32 last 793.76) - /unlink 30000 (time 1774520558.60 total 39.64 last 888.11) - /unlink 37186 (time 1774520568.61 total 49.64 last 718.58) - /unlink 40000 (time 1774520573.23 total 54.27 last 607.90) - /unlink 46930 (time 1774520583.23 total 64.27 last 693.00) - /unlink 50000 (time 1774520587.57 total 68.61 last 708.04) - /unlink 57524 (time 1774520597.57 total 78.61 last 752.36) - /unlink 60000 (time 1774520600.87 total 81.91 last 750.18) total: 64768 /unlink in 87.40 seconds: 741.02 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 4881 (time 1774520634.79 total 10.00 last 488.02) - open/close 9893 (time 1774520644.79 total 20.00 last 501.18) - open/close 14745 (time 1774520654.79 total 30.00 last 485.11) - open/close 19769 (time 1774520664.79 total 40.01 last 502.30) - open/close 24830 (time 1774520674.79 total 50.01 last 506.03) - open/close 29989 (time 1774520684.80 total 60.01 last 515.62) - open/close 35026 (time 1774520694.80 total 70.01 last 503.64) - open/close 40000 (time 1774520704.50 total 79.71 last 512.73) - open/close 45161 (time 1774520714.50 total 89.72 last 516.09) - open/close 50000 (time 1774520724.40 total 99.61 last 489.02) - open/close 54344 (time 1774520734.40 total 109.61 last 434.33) - open/close 58997 (time 1774520744.40 total 119.61 last 465.26) - open/close 64122 (time 1774520754.40 total 129.61 last 512.43) total: 64768 open/close in 131.01 seconds: 494.36 ops/second - /unlink 7913 (time 1774520767.12 total 10.00 last 791.25) - /unlink 10000 (time 1774520769.58 total 12.46 last 847.07) - /unlink 17680 (time 1774520779.58 total 22.46 last 767.98) - /unlink 20000 (time 1774520782.88 total 25.76 last 703.51) - /unlink 26874 (time 1774520792.88 total 35.77 last 687.20) - /unlink 30000 (time 1774520797.53 total 40.42 last 672.07) - /unlink 37095 (time 1774520807.53 total 50.42 last 709.43) - /unlink 40000 (time 1774520810.87 total 53.75 last 871.15) - /unlink 48062 (time 1774520820.87 total 63.75 last 806.15) - /unlink 50000 (time 1774520823.61 total 66.50 last 706.71) - /unlink 58164 (time 1774520833.61 total 76.50 last 816.35) total: 64768 /unlink in 84.05 seconds: 770.57 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg154-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg154-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg154-server stop mds service on oleg154-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg154-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing unload_modules_local oleg154-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg154-server: LNET ready to unload oleg154-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg154-client: oleg154-client: ssh exited with exit code 2 pdsh@oleg154-client: oleg154-server: ssh exited with exit code 2 pdsh@oleg154-client: oleg154-client: ssh exited with exit code 2 pdsh@oleg154-client: oleg154-server: ssh exited with exit code 2 PASS 106 (2861s) Stopping clients: oleg154-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg154-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg154-client: oleg154-server: ssh exited with exit code 2 oleg154-server: oleg154-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg154-server' oleg154-server: oleg154-server.virtnet: executing load_modules_local oleg154-server: Loading modules from /home/green/git/lustre-release/lustre oleg154-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg154-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg154-server: detected 4 online CPUs by sysfs oleg154-server: Force libcfs to create 2 CPU partitions oleg154-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg154-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3083 sec ========== 06:28:52 (1774520932)