== conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 00:57:48 (1769752668) Stopping clients: oleg347-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg347-client.virtnet /mnt/lustre2 (opts:-f) oleg347-server: oleg347-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg347-server' oleg347-server: oleg347-server.virtnet: executing load_modules_local oleg347-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg347-server: Loading modules from /home/green/git/lustre-release/lustre oleg347-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg347-server: detected 4 online CPUs by sysfs oleg347-server: MODOPTS_LIBCFS= oleg347-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg347-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg347-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg347-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg347-server: oleg347-server.virtnet: executing set_default_debug -1 all pdsh@oleg347-client: oleg347-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg347-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg347-client.virtnet: -o user_xattr,flock 192.168.203.147@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32648 392 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 675 (time 1769752796.40 total 10.00 last 67.49) - open/close 1393 (time 1769752806.41 total 20.02 last 71.68) - open/close 1999 (time 1769752816.43 total 30.03 last 60.52) - open/close 2695 (time 1769752826.44 total 40.05 last 69.47) - open/close 3437 (time 1769752836.45 total 50.05 last 74.18) - open/close 4212 (time 1769752846.46 total 60.06 last 77.43) - open/close 5045 (time 1769752856.47 total 70.08 last 83.18) - open/close 5821 (time 1769752866.47 total 80.08 last 77.57) - open/close 6557 (time 1769752876.49 total 90.09 last 73.50) - open/close 7203 (time 1769752886.49 total 100.10 last 64.57) - open/close 7809 (time 1769752896.51 total 110.12 last 60.49) - open/close 8529 (time 1769752906.51 total 120.12 last 71.97) - open/close 9464 (time 1769752916.53 total 130.13 last 93.37) - open/close 10000 (time 1769752922.23 total 135.84 last 94.00) - open/close 10765 (time 1769752932.23 total 145.84 last 76.48) - open/close 11591 (time 1769752942.24 total 155.84 last 82.56) - open/close 12433 (time 1769752952.24 total 165.85 last 84.17) - open/close 13228 (time 1769752962.24 total 175.85 last 79.49) - open/close 14094 (time 1769752972.25 total 185.86 last 86.54) - open/close 14978 (time 1769752982.25 total 195.86 last 88.40) - open/close 15746 (time 1769752992.26 total 205.86 last 76.75) - open/close 16508 (time 1769753002.26 total 215.87 last 76.16) - open/close 17118 (time 1769753012.27 total 225.88 last 60.95) - open/close 17912 (time 1769753022.27 total 235.88 last 79.38) - open/close 18697 (time 1769753032.28 total 245.89 last 78.44) - open/close 19471 (time 1769753042.29 total 255.89 last 77.34) - open/close 20000 (time 1769753048.29 total 261.89 last 88.18) - open/close 20827 (time 1769753058.29 total 271.90 last 82.68) - open/close 21650 (time 1769753068.29 total 281.90 last 82.28) - open/close 22544 (time 1769753078.30 total 291.90 last 89.36) - open/close 23260 (time 1769753088.31 total 301.92 last 71.49) - open/close 24115 (time 1769753098.32 total 311.93 last 85.39) - open/close 25044 (time 1769753108.33 total 321.93 last 92.89) - open/close 25929 (time 1769753118.33 total 331.94 last 88.45) - open/close 26652 (time 1769753128.34 total 341.95 last 72.23) - open/close 27474 (time 1769753138.35 total 351.95 last 82.17) - open/close 28290 (time 1769753148.40 total 362.00 last 81.17) - open/close 28984 (time 1769753158.40 total 372.01 last 69.37) - open/close 29676 (time 1769753168.41 total 382.01 last 69.17) - open/close 30000 (time 1769753172.48 total 386.09 last 79.55) - open/close 30771 (time 1769753182.48 total 396.09 last 77.09) - open/close 31613 (time 1769753192.49 total 406.10 last 84.12) - open/close 32490 (time 1769753202.49 total 416.10 last 87.68) - open/close 33247 (time 1769753212.50 total 426.10 last 75.67) - open/close 34144 (time 1769753222.54 total 436.15 last 89.31) - open/close 34811 (time 1769753232.54 total 446.15 last 66.68) - open/close 35706 (time 1769753242.55 total 456.16 last 89.44) - open/close 36605 (time 1769753252.56 total 466.16 last 89.85) - open/close 37444 (time 1769753262.56 total 476.16 last 83.90) - open/close 38357 (time 1769753272.56 total 486.16 last 91.29) - open/close 39179 (time 1769753282.58 total 496.18 last 82.04) - open/close 40000 (time 1769753292.28 total 505.88 last 84.66) - open/close 40894 (time 1769753302.28 total 515.89 last 89.37) - open/close 41533 (time 1769753312.28 total 525.89 last 63.89) - open/close 42354 (time 1769753322.28 total 535.89 last 82.07) - open/close 43172 (time 1769753332.30 total 545.91 last 81.66) - open/close 44001 (time 1769753342.31 total 555.91 last 82.86) - open/close 44837 (time 1769753352.31 total 565.92 last 83.58) - open/close 45749 (time 1769753362.31 total 575.92 last 91.18) - open/close 46504 (time 1769753372.31 total 585.92 last 75.49) - open/close 47375 (time 1769753382.32 total 595.93 last 87.04) - open/close 48118 (time 1769753392.33 total 605.93 last 74.24) - open/close 48997 (time 1769753402.33 total 615.93 last 87.89) - open/close 49742 (time 1769753412.33 total 625.94 last 74.48) - open/close 50000 (time 1769753415.91 total 629.51 last 72.18) - open/close 50648 (time 1769753425.91 total 639.52 last 64.76) - open/close 51377 (time 1769753435.91 total 649.52 last 72.88) - open/close 52287 (time 1769753445.92 total 659.52 last 90.99) - open/close 53147 (time 1769753455.92 total 669.53 last 85.94) - open/close 53992 (time 1769753465.93 total 679.54 last 84.41) - open/close 54775 (time 1769753475.97 total 689.58 last 78.01) - open/close 55546 (time 1769753485.98 total 699.58 last 77.06) - open/close 56431 (time 1769753495.98 total 709.58 last 88.50) - open/close 57342 (time 1769753505.98 total 719.59 last 91.02) - open/close 58200 (time 1769753515.99 total 729.59 last 85.80) - open/close 58959 (time 1769753526.00 total 739.60 last 75.80) - open/close 59789 (time 1769753536.01 total 749.62 last 82.86) - open/close 60000 (time 1769753538.47 total 752.07 last 85.98) - open/close 60836 (time 1769753548.48 total 762.08 last 83.54) - open/close 61678 (time 1769753558.49 total 772.09 last 84.11) - open/close 62503 (time 1769753568.49 total 782.09 last 82.48) - open/close 63425 (time 1769753578.51 total 792.11 last 92.02) - open/close 64248 (time 1769753588.51 total 802.12 last 82.25) total: 64768 open/close in 808.60 seconds: 80.10 ops/second - /unlink 784 (time 1769753609.23 total 10.01 last 78.35) - /unlink 1969 (time 1769753619.24 total 20.01 last 118.45) - /unlink 2824 (time 1769753629.24 total 30.01 last 85.49) - /unlink 3629 (time 1769753639.24 total 40.01 last 80.50) - /unlink 4418 (time 1769753649.25 total 50.02 last 78.82) - /unlink 5290 (time 1769753659.25 total 60.02 last 87.20) - /unlink 6068 (time 1769753669.25 total 70.03 last 77.76) - /unlink 6845 (time 1769753679.26 total 80.03 last 77.68) - /unlink 7543 (time 1769753689.26 total 90.04 last 69.75) - /unlink 8256 (time 1769753699.28 total 100.05 last 71.20) - /unlink 8978 (time 1769753709.29 total 110.07 last 72.08) - /unlink 9777 (time 1769753719.31 total 120.08 last 79.78) - /unlink 10000 (time 1769753722.13 total 122.91 last 78.96) - /unlink 10658 (time 1769753732.14 total 132.91 last 65.75) - /unlink 11426 (time 1769753742.18 total 142.95 last 76.50) - /unlink 11945 (time 1769753752.18 total 152.96 last 51.88) - /unlink 12623 (time 1769753762.19 total 162.97 last 67.74) - /unlink 13294 (time 1769753772.19 total 172.97 last 67.09) - /unlink 14106 (time 1769753782.20 total 182.97 last 81.14) - /unlink 14926 (time 1769753792.20 total 192.98 last 81.98) - /unlink 15674 (time 1769753802.21 total 202.98 last 74.78) - /unlink 16510 (time 1769753812.21 total 212.98 last 83.55) - /unlink 17222 (time 1769753822.21 total 222.99 last 71.20) - /unlink 18019 (time 1769753832.23 total 233.00 last 79.57) - /unlink 18781 (time 1769753842.24 total 243.01 last 76.12) - /unlink 19519 (time 1769753852.24 total 253.01 last 73.80) - /unlink 20000 (time 1769753859.62 total 260.39 last 65.17) - /unlink 20640 (time 1769753869.63 total 270.40 last 63.94) - /unlink 21434 (time 1769753879.64 total 280.41 last 79.33) - /unlink 22179 (time 1769753889.65 total 290.43 last 74.39) - /unlink 22813 (time 1769753899.66 total 300.44 last 63.35) - /unlink 23311 (time 1769753909.67 total 310.44 last 49.77) - /unlink 24107 (time 1769753919.69 total 320.46 last 79.44) - /unlink 24857 (time 1769753929.70 total 330.47 last 74.90) - /unlink 25614 (time 1769753939.70 total 340.47 last 75.69) - /unlink 26400 (time 1769753949.72 total 350.49 last 78.49) - /unlink 27278 (time 1769753959.72 total 360.49 last 87.77) - /unlink 28122 (time 1769753969.72 total 370.50 last 84.36) - /unlink 28947 (time 1769753979.73 total 380.50 last 82.47) - /unlink 29646 (time 1769753989.74 total 390.51 last 69.83) - /unlink 30000 (time 1769753995.16 total 395.93 last 65.33) - /unlink 30626 (time 1769754005.16 total 405.93 last 62.60) - /unlink 31277 (time 1769754015.16 total 415.93 last 65.08) - /unlink 31938 (time 1769754025.17 total 425.94 last 66.07) - /unlink 32572 (time 1769754035.17 total 435.94 last 63.39) - /unlink 33199 (time 1769754045.17 total 445.95 last 62.66) - /unlink 33861 (time 1769754055.18 total 455.95 last 66.14) - /unlink 34542 (time 1769754065.18 total 465.95 last 68.10) - /unlink 35428 (time 1769754075.18 total 475.96 last 88.58) - /unlink 36254 (time 1769754085.20 total 485.97 last 82.49) - /unlink 37107 (time 1769754095.20 total 495.97 last 85.29) - /unlink 37774 (time 1769754105.20 total 505.98 last 66.68) - /unlink 38522 (time 1769754115.21 total 515.98 last 74.78) - /unlink 39208 (time 1769754125.21 total 525.98 last 68.59) - /unlink 39834 (time 1769754135.23 total 536.01 last 62.43) - /unlink 40000 (time 1769754137.74 total 538.52 last 66.15) - /unlink 40761 (time 1769754147.76 total 548.53 last 75.96) - /unlink 41541 (time 1769754157.77 total 558.54 last 77.98) - /unlink 42345 (time 1769754167.77 total 568.54 last 80.38) - /unlink 43200 (time 1769754177.77 total 578.54 last 85.47) - /unlink 43852 (time 1769754187.78 total 588.55 last 65.16) - /unlink 44450 (time 1769754197.78 total 598.56 last 59.77) - /unlink 45109 (time 1769754207.81 total 608.58 last 65.71) - /unlink 45816 (time 1769754217.82 total 618.60 last 70.61) - /unlink 46476 (time 1769754227.83 total 628.60 last 65.96) - /unlink 47075 (time 1769754237.83 total 638.61 last 59.87) - /unlink 47737 (time 1769754247.85 total 648.62 last 66.13) - /unlink 48384 (time 1769754257.90 total 658.67 last 64.36) - /unlink 49051 (time 1769754267.91 total 668.68 last 66.63) - /unlink 49809 (time 1769754277.91 total 678.68 last 75.80) - /unlink 50000 (time 1769754280.31 total 681.08 last 79.69) - /unlink 50785 (time 1769754290.31 total 691.08 last 78.48) - /unlink 51593 (time 1769754300.31 total 701.09 last 80.76) - /unlink 52398 (time 1769754310.32 total 711.09 last 80.47) - /unlink 53228 (time 1769754320.32 total 721.09 last 82.96) - /unlink 54147 (time 1769754330.33 total 731.10 last 91.86) - /unlink 55006 (time 1769754340.34 total 741.11 last 85.77) - /unlink 55856 (time 1769754350.36 total 751.13 last 84.88) - /unlink 56594 (time 1769754360.36 total 761.13 last 73.79) - /unlink 57413 (time 1769754370.36 total 771.13 last 81.88) - /unlink 58289 (time 1769754380.36 total 781.13 last 87.59) - /unlink 59017 (time 1769754390.40 total 791.17 last 72.50) - /unlink 59623 (time 1769754400.40 total 801.18 last 60.58) - /unlink 60000 (time 1769754406.57 total 807.34 last 61.13) - /unlink 60591 (time 1769754416.58 total 817.35 last 59.06) - /unlink 61439 (time 1769754426.58 total 827.36 last 84.76) - /unlink 62219 (time 1769754436.60 total 837.37 last 77.90) - /unlink 63022 (time 1769754446.61 total 847.39 last 80.15) - /unlink 63921 (time 1769754456.62 total 857.40 last 89.82) - /unlink 64678 (time 1769754466.64 total 867.41 last 75.61) total: 64768 /unlink in 868.51 seconds: 74.57 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 808 (time 1769754505.55 total 10.04 last 80.50) - open/close 1702 (time 1769754515.56 total 20.04 last 89.40) - open/close 2518 (time 1769754525.56 total 30.05 last 81.53) - open/close 3297 (time 1769754535.56 total 40.05 last 77.89) - open/close 4169 (time 1769754545.57 total 50.05 last 87.14) - open/close 4949 (time 1769754555.58 total 60.06 last 77.96) - open/close 5839 (time 1769754565.59 total 70.07 last 88.93) - open/close 6568 (time 1769754575.60 total 80.08 last 72.77) - open/close 7219 (time 1769754585.61 total 90.09 last 65.06) - open/close 7942 (time 1769754595.62 total 100.10 last 72.25) - open/close 8774 (time 1769754605.62 total 110.10 last 83.16) - open/close 9680 (time 1769754615.63 total 120.11 last 90.50) - open/close 10000 (time 1769754619.25 total 123.73 last 88.52) - open/close 10834 (time 1769754629.25 total 133.74 last 83.34) - open/close 11631 (time 1769754639.26 total 143.74 last 79.66) - open/close 12397 (time 1769754649.26 total 153.74 last 76.59) - open/close 13274 (time 1769754659.26 total 163.74 last 87.67) - open/close 14092 (time 1769754669.26 total 173.75 last 81.79) - open/close 14933 (time 1769754679.28 total 183.76 last 83.97) - open/close 15755 (time 1769754689.28 total 193.77 last 82.17) - open/close 16630 (time 1769754699.29 total 203.77 last 87.46) - open/close 17493 (time 1769754709.29 total 213.77 last 86.30) - open/close 18333 (time 1769754719.29 total 223.77 last 83.97) - open/close 19150 (time 1769754729.30 total 233.78 last 81.63) - open/close 19974 (time 1769754739.30 total 243.78 last 82.38) - open/close 20733 (time 1769754749.31 total 253.79 last 75.85) - open/close 21593 (time 1769754759.32 total 263.80 last 85.91) - open/close 22362 (time 1769754769.33 total 273.81 last 76.83) - open/close 23044 (time 1769754779.35 total 283.83 last 68.06) - open/close 23836 (time 1769754789.35 total 293.83 last 79.19) - open/close 24696 (time 1769754799.35 total 303.83 last 85.99) - open/close 25493 (time 1769754809.36 total 313.84 last 79.61) - open/close 26262 (time 1769754819.36 total 323.84 last 76.90) - open/close 26930 (time 1769754829.38 total 333.86 last 66.68) - open/close 27522 (time 1769754839.40 total 343.88 last 59.10) - open/close 28196 (time 1769754849.40 total 353.88 last 67.36) - open/close 28969 (time 1769754859.40 total 363.88 last 77.29) - open/close 29674 (time 1769754869.41 total 373.89 last 70.47) - open/close 30000 (time 1769754874.00 total 378.48 last 71.04) - open/close 30767 (time 1769754884.02 total 388.50 last 76.50) - open/close 31618 (time 1769754894.05 total 398.53 last 84.87) - open/close 32448 (time 1769754904.05 total 408.53 last 82.99) - open/close 33242 (time 1769754914.07 total 418.55 last 79.23) - open/close 34014 (time 1769754924.08 total 428.56 last 77.14) - open/close 34835 (time 1769754934.08 total 438.56 last 82.07) - open/close 35561 (time 1769754944.09 total 448.57 last 72.55) - open/close 36259 (time 1769754954.11 total 458.59 last 69.68) - open/close 36832 (time 1769754964.12 total 468.60 last 57.23) - open/close 37700 (time 1769754974.12 total 478.60 last 86.78) - open/close 38563 (time 1769754984.14 total 488.62 last 86.16) - open/close 39341 (time 1769754994.18 total 498.66 last 77.48) - open/close 39942 (time 1769755004.18 total 508.67 last 60.06) - open/close 40548 (time 1769755014.19 total 518.67 last 60.59) - open/close 41002 (time 1769755024.20 total 528.68 last 45.33) - open/close 41710 (time 1769755034.21 total 538.69 last 70.75) - open/close 42413 (time 1769755044.22 total 548.70 last 70.24) - open/close 43263 (time 1769755054.22 total 558.70 last 84.99) - open/close 44022 (time 1769755064.26 total 568.74 last 75.59) - open/close 44813 (time 1769755074.27 total 578.75 last 79.03) - open/close 45625 (time 1769755084.28 total 588.76 last 81.13) - open/close 46344 (time 1769755094.29 total 598.77 last 71.80) - open/close 47088 (time 1769755104.30 total 608.78 last 74.32) - open/close 47872 (time 1769755114.30 total 618.78 last 78.39) - open/close 48512 (time 1769755124.31 total 628.79 last 63.97) - open/close 49152 (time 1769755134.32 total 638.80 last 63.91) - open/close 49904 (time 1769755144.33 total 648.82 last 75.09) - open/close 50681 (time 1769755154.35 total 658.83 last 77.59) - open/close 51447 (time 1769755164.35 total 668.83 last 76.56) - open/close 52166 (time 1769755174.36 total 678.84 last 71.85) - open/close 52890 (time 1769755184.36 total 688.84 last 72.38) - open/close 53576 (time 1769755194.37 total 698.85 last 68.54) - open/close 54260 (time 1769755204.37 total 708.85 last 68.40) - open/close 55007 (time 1769755214.38 total 718.86 last 74.63) - open/close 55601 (time 1769755224.38 total 728.86 last 59.40) - open/close 56313 (time 1769755234.38 total 738.86 last 71.19) - open/close 56979 (time 1769755244.40 total 748.88 last 66.51) - open/close 57689 (time 1769755254.40 total 758.88 last 70.96) - open/close 58354 (time 1769755264.40 total 768.89 last 66.47) - open/close 59100 (time 1769755274.42 total 778.90 last 74.49) - open/close 59846 (time 1769755284.43 total 788.91 last 74.55) - open/close 60000 (time 1769755286.69 total 791.17 last 68.06) - open/close 60883 (time 1769755296.70 total 801.18 last 88.24) - open/close 61815 (time 1769755306.70 total 811.19 last 93.12) - open/close 62655 (time 1769755316.73 total 821.21 last 83.82) - open/close 63471 (time 1769755326.73 total 831.21 last 81.60) - open/close 64260 (time 1769755336.73 total 841.21 last 78.89) total: 64768 open/close in 847.45 seconds: 76.43 ops/second - /unlink 757 (time 1769755356.87 total 10.00 last 75.68) - /unlink 1443 (time 1769755366.88 total 20.01 last 68.58) - /unlink 2207 (time 1769755376.88 total 30.01 last 76.39) - /unlink 3020 (time 1769755386.88 total 40.01 last 81.24) - /unlink 3756 (time 1769755396.90 total 50.03 last 73.50) - /unlink 4454 (time 1769755406.91 total 60.04 last 69.74) - /unlink 5238 (time 1769755416.91 total 70.04 last 78.40) - /unlink 5997 (time 1769755426.91 total 80.04 last 75.85) - /unlink 6806 (time 1769755436.94 total 90.07 last 80.66) - /unlink 7705 (time 1769755446.95 total 100.08 last 89.83) - /unlink 8485 (time 1769755456.95 total 110.08 last 77.98) - /unlink 9195 (time 1769755466.96 total 120.09 last 70.93) - /unlink 9819 (time 1769755476.97 total 130.10 last 62.35) - /unlink 10000 (time 1769755479.48 total 132.61 last 72.21) - /unlink 10803 (time 1769755489.49 total 142.62 last 80.23) - /unlink 11518 (time 1769755499.49 total 152.62 last 71.47) - /unlink 12235 (time 1769755509.50 total 162.63 last 71.63) - /unlink 13002 (time 1769755519.51 total 172.64 last 76.65) - /unlink 13694 (time 1769755529.51 total 182.64 last 69.14) - /unlink 14288 (time 1769755539.53 total 192.66 last 59.30) - /unlink 14990 (time 1769755549.60 total 202.73 last 69.74) - /unlink 15794 (time 1769755559.60 total 212.73 last 80.39) - /unlink 16741 (time 1769755569.61 total 222.74 last 94.64) - /unlink 17561 (time 1769755579.61 total 232.74 last 82.00) - /unlink 18453 (time 1769755589.61 total 242.74 last 89.16) - /unlink 19369 (time 1769755599.62 total 252.75 last 91.54) - /unlink 20000 (time 1769755607.51 total 260.64 last 79.94) - /unlink 20728 (time 1769755617.52 total 270.65 last 72.73) - /unlink 21587 (time 1769755627.53 total 280.66 last 85.85) - /unlink 22450 (time 1769755637.53 total 290.66 last 86.29) - /unlink 23178 (time 1769755647.53 total 300.66 last 72.77) - /unlink 23949 (time 1769755657.53 total 310.66 last 77.08) - /unlink 24734 (time 1769755667.54 total 320.67 last 78.48) - /unlink 25562 (time 1769755677.54 total 330.67 last 82.76) - /unlink 26407 (time 1769755687.55 total 340.68 last 84.40) - /unlink 27242 (time 1769755697.56 total 350.69 last 83.46) - /unlink 28126 (time 1769755707.56 total 360.69 last 88.38) - /unlink 28916 (time 1769755717.57 total 370.70 last 78.96) - /unlink 29663 (time 1769755727.57 total 380.70 last 74.63) - /unlink 30000 (time 1769755731.70 total 384.83 last 81.66) - /unlink 30935 (time 1769755741.72 total 394.85 last 93.33) - /unlink 31836 (time 1769755751.73 total 404.86 last 90.00) - /unlink 32702 (time 1769755761.73 total 414.86 last 86.58) - /unlink 33445 (time 1769755771.74 total 424.87 last 74.26) - /unlink 34218 (time 1769755781.75 total 434.88 last 77.22) - /unlink 35083 (time 1769755791.75 total 444.88 last 86.46) - /unlink 35886 (time 1769755801.76 total 454.89 last 80.21) - /unlink 36732 (time 1769755811.77 total 464.90 last 84.51) - /unlink 37584 (time 1769755821.78 total 474.91 last 85.14) - /unlink 38329 (time 1769755831.78 total 484.91 last 74.48) - /unlink 39171 (time 1769755841.79 total 494.92 last 84.14) - /unlink 39873 (time 1769755851.80 total 504.93 last 70.14) - /unlink 40708 (time 1769755861.81 total 514.94 last 83.45) - /unlink 41301 (time 1769755871.81 total 524.94 last 59.28) - /unlink 42049 (time 1769755881.82 total 534.95 last 74.73) - /unlink 42734 (time 1769755891.83 total 544.96 last 68.44) - /unlink 43524 (time 1769755901.83 total 554.96 last 78.98) - /unlink 44374 (time 1769755911.83 total 564.96 last 84.96) - /unlink 45160 (time 1769755921.84 total 574.97 last 78.52) - /unlink 45920 (time 1769755931.85 total 584.98 last 75.93) - /unlink 46745 (time 1769755941.86 total 594.99 last 82.48) - /unlink 47555 (time 1769755951.87 total 605.00 last 80.89) - /unlink 48419 (time 1769755961.88 total 615.01 last 86.35) - /unlink 49294 (time 1769755971.88 total 625.01 last 87.44) - /unlink 50000 (time 1769755979.45 total 632.58 last 93.27) - /unlink 50932 (time 1769755989.46 total 642.59 last 93.16) - /unlink 51738 (time 1769755999.46 total 652.59 last 80.57) - /unlink 52594 (time 1769756009.46 total 662.59 last 85.58) - /unlink 53382 (time 1769756019.48 total 672.61 last 78.70) - /unlink 54205 (time 1769756029.48 total 682.61 last 82.27) - /unlink 55065 (time 1769756039.50 total 692.63 last 85.83) - /unlink 55996 (time 1769756049.50 total 702.63 last 93.07) - /unlink 56866 (time 1769756059.51 total 712.64 last 86.95) - /unlink 57702 (time 1769756069.51 total 722.64 last 83.54) - /unlink 58554 (time 1769756079.53 total 732.66 last 85.04) - /unlink 59467 (time 1769756089.53 total 742.66 last 91.30) - /unlink 60000 (time 1769756096.39 total 749.52 last 77.71) - /unlink 60877 (time 1769756106.39 total 759.52 last 87.69) - /unlink 61791 (time 1769756116.40 total 769.53 last 91.36) - /unlink 62668 (time 1769756126.40 total 779.53 last 87.70) - /unlink 63623 (time 1769756136.40 total 789.53 last 95.45) - /unlink 64493 (time 1769756146.42 total 799.55 last 86.87) total: 64768 /unlink in 802.11 seconds: 80.75 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 949 (time 1769756184.45 total 10.01 last 94.79) - open/close 1943 (time 1769756194.45 total 20.01 last 99.39) - open/close 2837 (time 1769756204.46 total 30.02 last 89.33) - open/close 3716 (time 1769756214.46 total 40.02 last 87.89) - open/close 4599 (time 1769756224.47 total 50.03 last 88.24) - open/close 5463 (time 1769756234.47 total 60.03 last 86.36) - open/close 6287 (time 1769756244.48 total 70.04 last 82.35) - open/close 7101 (time 1769756254.48 total 80.04 last 81.39) - open/close 7904 (time 1769756264.49 total 90.05 last 80.25) - open/close 8874 (time 1769756274.49 total 100.05 last 96.98) - open/close 9909 (time 1769756284.50 total 110.06 last 103.40) - open/close 10895 (time 1769756294.51 total 120.07 last 98.50) - open/close 11827 (time 1769756304.51 total 130.07 last 93.19) - open/close 12855 (time 1769756314.51 total 140.07 last 102.77) - open/close 13731 (time 1769756324.53 total 150.09 last 87.47) - open/close 14652 (time 1769756334.53 total 160.09 last 92.08) - open/close 15634 (time 1769756344.54 total 170.10 last 98.08) - open/close 16568 (time 1769756354.55 total 180.11 last 93.31) - open/close 17451 (time 1769756364.56 total 190.12 last 88.22) - open/close 18164 (time 1769756374.58 total 200.14 last 71.18) - open/close 18780 (time 1769756384.59 total 210.15 last 61.53) - open/close 19756 (time 1769756394.59 total 220.15 last 97.58) - open/close 20000 (time 1769756397.96 total 223.52 last 72.46) - open/close 20818 (time 1769756407.96 total 233.52 last 81.77) - open/close 21660 (time 1769756417.97 total 243.53 last 84.16) - open/close 22543 (time 1769756427.97 total 253.53 last 88.27) - open/close 23337 (time 1769756437.97 total 263.53 last 79.37) - open/close 24235 (time 1769756447.97 total 273.53 last 89.79) - open/close 25008 (time 1769756457.98 total 283.54 last 77.22) - open/close 25937 (time 1769756468.00 total 293.56 last 92.77) - open/close 26915 (time 1769756478.01 total 303.57 last 97.71) - open/close 27696 (time 1769756488.03 total 313.59 last 77.90) - open/close 28383 (time 1769756498.04 total 323.60 last 68.63) - open/close 28992 (time 1769756508.05 total 333.61 last 60.85) - open/close 29756 (time 1769756518.06 total 343.61 last 76.37) - open/close 30000 (time 1769756520.63 total 346.19 last 94.92) - open/close 30853 (time 1769756530.65 total 356.21 last 85.10) - open/close 31615 (time 1769756540.65 total 366.21 last 76.17) - open/close 32524 (time 1769756550.66 total 376.22 last 90.83) - open/close 33361 (time 1769756560.66 total 386.22 last 83.67) - open/close 34226 (time 1769756570.67 total 396.23 last 86.42) - open/close 35125 (time 1769756580.67 total 406.23 last 89.90) - open/close 35911 (time 1769756590.68 total 416.24 last 78.58) - open/close 36676 (time 1769756600.69 total 426.24 last 76.43) - open/close 37349 (time 1769756610.69 total 436.25 last 67.27) - open/close 38003 (time 1769756620.70 total 446.26 last 65.32) - open/close 38692 (time 1769756630.75 total 456.30 last 68.61) - open/close 39536 (time 1769756640.75 total 466.31 last 84.33) - open/close 40000 (time 1769756646.78 total 472.34 last 76.99) - open/close 40847 (time 1769756656.78 total 482.34 last 84.66) - open/close 41607 (time 1769756666.79 total 492.35 last 75.95) - open/close 42301 (time 1769756676.79 total 502.35 last 69.39) - open/close 42989 (time 1769756686.79 total 512.35 last 68.79) - open/close 43865 (time 1769756696.80 total 522.36 last 87.57) - open/close 44658 (time 1769756706.80 total 532.36 last 79.25) - open/close 45476 (time 1769756716.81 total 542.37 last 81.72) - open/close 46341 (time 1769756726.83 total 552.39 last 86.33) - open/close 47244 (time 1769756736.85 total 562.40 last 90.19) - open/close 48032 (time 1769756746.85 total 572.41 last 78.78) - open/close 48803 (time 1769756756.85 total 582.41 last 77.08) - open/close 49597 (time 1769756766.86 total 592.41 last 79.36) - open/close 50000 (time 1769756772.23 total 597.79 last 74.98) - open/close 50715 (time 1769756782.24 total 607.80 last 71.43) - open/close 51495 (time 1769756792.25 total 617.81 last 77.90) - open/close 52199 (time 1769756802.26 total 627.82 last 70.34) - open/close 52824 (time 1769756812.27 total 637.83 last 62.42) - open/close 53626 (time 1769756822.28 total 647.84 last 80.15) - open/close 54504 (time 1769756832.28 total 657.84 last 87.78) - open/close 55218 (time 1769756842.29 total 667.85 last 71.33) - open/close 55976 (time 1769756852.29 total 677.85 last 75.79) - open/close 56863 (time 1769756862.30 total 687.86 last 88.66) - open/close 57652 (time 1769756872.30 total 697.86 last 78.88) - open/close 58209 (time 1769756882.30 total 707.86 last 55.67) - open/close 59074 (time 1769756892.32 total 717.87 last 86.41) - open/close 59853 (time 1769756902.33 total 727.89 last 77.77) - open/close 60677 (time 1769756912.37 total 737.93 last 82.11) - open/close 61512 (time 1769756922.38 total 747.94 last 83.40) - open/close 62269 (time 1769756932.39 total 757.94 last 75.65) - open/close 63211 (time 1769756942.39 total 767.95 last 94.19) - open/close 63985 (time 1769756952.39 total 777.95 last 77.35) total: 64768 open/close in 787.88 seconds: 82.21 ops/second - /unlink 869 (time 1769756976.15 total 10.01 last 86.81) - /unlink 1540 (time 1769756986.16 total 20.02 last 67.03) - /unlink 2235 (time 1769756996.17 total 30.03 last 69.42) - /unlink 3001 (time 1769757006.17 total 40.04 last 76.57) - /unlink 3854 (time 1769757016.18 total 50.04 last 85.25) - /unlink 4737 (time 1769757026.18 total 60.05 last 88.27) - /unlink 5580 (time 1769757036.19 total 70.05 last 84.25) - /unlink 6394 (time 1769757046.20 total 80.06 last 81.31) - /unlink 7233 (time 1769757056.20 total 90.06 last 83.89) - /unlink 7974 (time 1769757066.20 total 100.06 last 74.09) - /unlink 8815 (time 1769757076.21 total 110.07 last 84.07) - /unlink 9730 (time 1769757086.22 total 120.08 last 91.40) - /unlink 10000 (time 1769757089.42 total 123.28 last 84.40) - /unlink 10856 (time 1769757099.42 total 133.29 last 85.54) - /unlink 11686 (time 1769757109.43 total 143.29 last 82.95) - /unlink 12591 (time 1769757119.44 total 153.31 last 90.37) - /unlink 13299 (time 1769757129.45 total 163.31 last 70.74) - /unlink 14092 (time 1769757139.45 total 173.31 last 79.30) - /unlink 14901 (time 1769757149.46 total 183.33 last 80.81) - /unlink 15774 (time 1769757159.47 total 193.33 last 87.28) - /unlink 16582 (time 1769757169.47 total 203.34 last 80.73) - /unlink 17434 (time 1769757179.48 total 213.34 last 85.13) - /unlink 18252 (time 1769757189.49 total 223.36 last 81.70) - /unlink 19142 (time 1769757199.51 total 233.37 last 88.87) - /unlink 19982 (time 1769757209.51 total 243.37 last 84.00) - /unlink 20866 (time 1769757219.52 total 253.39 last 88.29) - /unlink 21714 (time 1769757229.53 total 263.39 last 84.78) - /unlink 22543 (time 1769757239.54 total 273.40 last 82.81) - /unlink 23381 (time 1769757249.54 total 283.40 last 83.78) - /unlink 24205 (time 1769757259.55 total 293.41 last 82.33) - /unlink 24939 (time 1769757269.57 total 303.43 last 73.25) - /unlink 25800 (time 1769757279.57 total 313.43 last 86.07) - /unlink 26724 (time 1769757289.58 total 323.44 last 92.34) - /unlink 27684 (time 1769757299.58 total 333.44 last 95.97) - /unlink 28547 (time 1769757309.60 total 343.46 last 86.18) - /unlink 29288 (time 1769757319.61 total 353.47 last 73.99) - /unlink 30000 (time 1769757327.88 total 361.74 last 86.07) - /unlink 30824 (time 1769757337.88 total 371.75 last 82.39) - /unlink 31760 (time 1769757347.88 total 381.75 last 93.59) - /unlink 32666 (time 1769757357.89 total 391.75 last 90.57) - /unlink 33513 (time 1769757367.90 total 401.76 last 84.61) - /unlink 34275 (time 1769757377.90 total 411.76 last 76.19) - /unlink 35076 (time 1769757387.90 total 421.76 last 80.09) - /unlink 35876 (time 1769757397.95 total 431.81 last 79.60) - /unlink 36597 (time 1769757407.99 total 441.85 last 71.81) - /unlink 37275 (time 1769757418.01 total 451.87 last 67.67) - /unlink 38059 (time 1769757428.03 total 461.89 last 78.23) - /unlink 38803 (time 1769757438.04 total 471.90 last 74.34) - /unlink 39671 (time 1769757448.04 total 481.91 last 86.78) - /unlink 40000 (time 1769757451.85 total 485.71 last 86.44) - /unlink 40833 (time 1769757461.86 total 495.72 last 83.21) - /unlink 41743 (time 1769757471.86 total 505.72 last 90.98) - /unlink 42589 (time 1769757481.88 total 515.74 last 84.46) - /unlink 43512 (time 1769757491.88 total 525.74 last 92.28) - /unlink 44364 (time 1769757501.88 total 535.75 last 85.16) - /unlink 45165 (time 1769757511.90 total 545.76 last 79.96) - /unlink 45952 (time 1769757521.91 total 555.77 last 78.65) - /unlink 46756 (time 1769757531.91 total 565.78 last 80.34) - /unlink 47590 (time 1769757541.93 total 575.80 last 83.24) - /unlink 48367 (time 1769757551.94 total 585.80 last 77.68) - /unlink 49171 (time 1769757561.94 total 595.80 last 80.39) - /unlink 50000 (time 1769757570.54 total 604.40 last 96.37) - /unlink 50902 (time 1769757580.54 total 614.40 last 90.19) - /unlink 51718 (time 1769757590.55 total 624.41 last 81.54) - /unlink 52568 (time 1769757600.55 total 634.41 last 84.99) - /unlink 53394 (time 1769757610.56 total 644.42 last 82.50) - /unlink 54262 (time 1769757620.56 total 654.43 last 86.78) - /unlink 55125 (time 1769757630.57 total 664.43 last 86.28) - /unlink 55974 (time 1769757640.57 total 674.44 last 84.85) - /unlink 56832 (time 1769757650.58 total 684.44 last 85.78) - /unlink 57741 (time 1769757660.59 total 694.45 last 90.80) - /unlink 58599 (time 1769757670.59 total 704.45 last 85.79) - /unlink 59459 (time 1769757680.59 total 714.46 last 85.96) - /unlink 60000 (time 1769757687.18 total 721.04 last 82.15) - /unlink 60900 (time 1769757697.18 total 731.04 last 89.97) - /unlink 61766 (time 1769757707.19 total 741.05 last 86.55) - /unlink 62571 (time 1769757717.20 total 751.06 last 80.44) - /unlink 63295 (time 1769757727.21 total 761.07 last 72.33) - /unlink 64156 (time 1769757737.21 total 771.08 last 86.03) total: 64768 /unlink in 776.86 seconds: 83.37 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1017 (time 1769757779.20 total 10.01 last 101.64) - open/close 1951 (time 1769757789.21 total 20.01 last 93.36) - open/close 2981 (time 1769757799.22 total 30.02 last 102.92) - open/close 3852 (time 1769757809.22 total 40.02 last 87.08) - open/close 4824 (time 1769757819.23 total 50.03 last 97.14) - open/close 5774 (time 1769757829.23 total 60.03 last 95.00) - open/close 6719 (time 1769757839.23 total 70.03 last 94.47) - open/close 7764 (time 1769757849.24 total 80.04 last 104.42) - open/close 8702 (time 1769757859.24 total 90.04 last 93.78) - open/close 9619 (time 1769757869.24 total 100.04 last 91.69) - open/close 10000 (time 1769757873.28 total 104.09 last 94.21) - open/close 10969 (time 1769757883.29 total 114.09 last 96.84) - open/close 11975 (time 1769757893.30 total 124.10 last 100.47) - open/close 12913 (time 1769757903.31 total 134.11 last 93.72) - open/close 13833 (time 1769757913.32 total 144.12 last 91.93) - open/close 14792 (time 1769757923.32 total 154.12 last 95.88) - open/close 15727 (time 1769757933.33 total 164.13 last 93.40) - open/close 16600 (time 1769757943.34 total 174.14 last 87.22) - open/close 17477 (time 1769757953.34 total 184.14 last 87.67) - open/close 18436 (time 1769757963.35 total 194.15 last 95.84) - open/close 19249 (time 1769757973.35 total 204.15 last 81.29) - open/close 20000 (time 1769757980.93 total 211.73 last 99.10) - open/close 20974 (time 1769757990.94 total 221.74 last 97.34) - open/close 22065 (time 1769758000.94 total 231.74 last 109.01) - open/close 23118 (time 1769758010.94 total 241.75 last 105.30) - open/close 24027 (time 1769758020.94 total 251.75 last 90.89) - open/close 25137 (time 1769758030.95 total 261.75 last 110.97) - open/close 26178 (time 1769758040.96 total 271.76 last 103.96) - open/close 27110 (time 1769758050.96 total 281.77 last 93.17) - open/close 28040 (time 1769758060.97 total 291.77 last 92.95) - open/close 29069 (time 1769758070.97 total 301.78 last 102.85) - open/close 29977 (time 1769758080.98 total 311.78 last 90.76) - open/close 30741 (time 1769758090.99 total 321.79 last 76.34) - open/close 31707 (time 1769758100.99 total 331.79 last 96.59) - open/close 32759 (time 1769758110.99 total 341.80 last 105.13) - open/close 33841 (time 1769758121.00 total 351.80 last 108.12) - open/close 34977 (time 1769758131.01 total 361.81 last 113.49) - open/close 36065 (time 1769758141.01 total 371.81 last 108.80) - open/close 37030 (time 1769758151.02 total 381.82 last 96.45) - open/close 38124 (time 1769758161.02 total 391.82 last 109.36) - open/close 39247 (time 1769758171.05 total 401.85 last 112.00) - open/close 40000 (time 1769758177.25 total 408.05 last 121.39) - open/close 41172 (time 1769758187.25 total 418.05 last 117.19) - open/close 42063 (time 1769758197.26 total 428.06 last 89.01) - open/close 43132 (time 1769758207.26 total 438.06 last 106.88) - open/close 44245 (time 1769758217.27 total 448.07 last 111.27) - open/close 45308 (time 1769758227.27 total 458.07 last 106.26) - open/close 46338 (time 1769758237.27 total 468.07 last 102.99) - open/close 47365 (time 1769758247.29 total 478.09 last 102.50) - open/close 48402 (time 1769758257.30 total 488.10 last 103.61) - open/close 49406 (time 1769758267.30 total 498.10 last 100.38) - open/close 50000 (time 1769758272.61 total 503.42 last 111.81) - open/close 51097 (time 1769758282.61 total 513.42 last 109.69) - open/close 52181 (time 1769758292.62 total 523.42 last 108.37) - open/close 53301 (time 1769758302.62 total 533.42 last 112.00) - open/close 54437 (time 1769758312.62 total 543.42 last 113.60) - open/close 55333 (time 1769758322.63 total 553.43 last 89.48) - open/close 56331 (time 1769758332.63 total 563.43 last 99.78) - open/close 57954 (time 1769758342.63 total 573.44 last 162.29) - open/close 59316 (time 1769758352.64 total 583.45 last 136.05) - open/close 60000 (time 1769758358.92 total 589.72 last 109.05) - open/close 61180 (time 1769758368.93 total 599.73 last 117.88) - open/close 62222 (time 1769758378.94 total 609.74 last 104.09) - open/close 63238 (time 1769758388.94 total 619.74 last 101.58) - open/close 64394 (time 1769758398.94 total 629.74 last 115.58) total: 64768 open/close in 633.17 seconds: 102.29 ops/second - /unlink 1166 (time 1769758415.00 total 10.00 last 116.55) - /unlink 2216 (time 1769758425.01 total 20.01 last 104.95) - /unlink 3312 (time 1769758435.01 total 30.02 last 109.51) - /unlink 4421 (time 1769758445.02 total 40.02 last 110.82) - /unlink 5523 (time 1769758455.02 total 50.02 last 110.20) - /unlink 6779 (time 1769758465.03 total 60.03 last 125.55) - /unlink 7927 (time 1769758475.03 total 70.03 last 114.79) - /unlink 9045 (time 1769758485.05 total 80.05 last 111.58) - /unlink 10000 (time 1769758493.29 total 88.29 last 115.90) - /unlink 11287 (time 1769758503.29 total 98.29 last 128.66) - /unlink 12539 (time 1769758513.30 total 108.30 last 125.08) - /unlink 13813 (time 1769758523.30 total 118.31 last 127.33) - /unlink 14944 (time 1769758533.31 total 128.31 last 113.06) - /unlink 16049 (time 1769758543.31 total 138.32 last 110.43) - /unlink 17181 (time 1769758553.32 total 148.32 last 113.15) - /unlink 18487 (time 1769758563.32 total 158.32 last 130.60) - /unlink 20000 (time 1769758570.17 total 165.17 last 220.90) - /unlink 22169 (time 1769758580.18 total 175.18 last 216.72) - /unlink 24026 (time 1769758590.18 total 185.18 last 185.69) - /unlink 26153 (time 1769758600.18 total 195.18 last 212.65) - /unlink 28531 (time 1769758610.18 total 205.18 last 237.73) - /unlink 30000 (time 1769758616.21 total 211.21 last 243.86) - /unlink 32178 (time 1769758626.21 total 221.21 last 217.79) - /unlink 34491 (time 1769758636.21 total 231.21 last 231.23) - /unlink 36627 (time 1769758646.21 total 241.21 last 213.56) - /unlink 39645 (time 1769758656.21 total 251.21 last 301.75) - /unlink 42605 (time 1769758666.21 total 261.22 last 295.94) - /unlink 45590 (time 1769758676.22 total 271.22 last 298.45) - /unlink 48087 (time 1769758686.22 total 281.22 last 249.69) - /unlink 50000 (time 1769758692.91 total 287.91 last 285.91) - /unlink 52294 (time 1769758702.91 total 297.91 last 229.34) - /unlink 55222 (time 1769758712.91 total 307.92 last 292.67) - /unlink 57940 (time 1769758722.92 total 317.92 last 271.65) - /unlink 60000 (time 1769758731.24 total 326.24 last 247.64) - /unlink 62154 (time 1769758741.24 total 336.24 last 215.40) - /unlink 64749 (time 1769758751.24 total 346.24 last 259.48) total: 64768 /unlink in 346.33 seconds: 187.01 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1342 (time 1769758781.61 total 10.00 last 134.17) - open/close 2816 (time 1769758791.61 total 20.01 last 147.34) - open/close 4384 (time 1769758801.61 total 30.01 last 156.74) - open/close 5750 (time 1769758811.61 total 40.01 last 136.58) - open/close 7248 (time 1769758821.62 total 50.01 last 149.76) - open/close 8907 (time 1769758831.62 total 60.02 last 165.79) - open/close 10000 (time 1769758840.47 total 68.86 last 123.60) - open/close 11461 (time 1769758850.47 total 78.87 last 146.05) - open/close 12736 (time 1769758860.48 total 88.87 last 127.42) - open/close 13833 (time 1769758870.48 total 98.88 last 109.62) - open/close 15194 (time 1769758880.49 total 108.89 last 136.03) - open/close 16614 (time 1769758890.49 total 118.89 last 141.94) - open/close 18373 (time 1769758900.50 total 128.90 last 175.79) - open/close 19999 (time 1769758910.50 total 138.90 last 162.58) - open/close 21742 (time 1769758920.51 total 148.90 last 174.22) - open/close 23139 (time 1769758930.52 total 158.91 last 139.56) - open/close 24871 (time 1769758940.52 total 168.91 last 173.16) - open/close 26295 (time 1769758950.52 total 178.91 last 142.39) - open/close 27756 (time 1769758960.52 total 188.92 last 146.09) - open/close 29462 (time 1769758970.53 total 198.92 last 170.47) - open/close 30000 (time 1769758973.34 total 201.73 last 191.47) - open/close 31744 (time 1769758983.34 total 211.74 last 174.27) - open/close 33355 (time 1769758993.35 total 221.75 last 161.02) - open/close 34970 (time 1769759003.35 total 231.75 last 161.47) - open/close 36281 (time 1769759013.36 total 241.76 last 131.00) - open/close 37782 (time 1769759023.36 total 251.76 last 150.05) - open/close 39436 (time 1769759033.36 total 261.76 last 165.39) - open/close 40000 (time 1769759036.21 total 264.60 last 198.33) - open/close 41637 (time 1769759046.21 total 274.61 last 163.67) - open/close 43734 (time 1769759056.21 total 284.61 last 209.68) - open/close 45784 (time 1769759066.21 total 294.61 last 205.00) - open/close 47591 (time 1769759076.21 total 304.61 last 180.68) - open/close 49312 (time 1769759086.21 total 314.61 last 172.09) - open/close 50000 (time 1769759090.15 total 318.55 last 174.73) - open/close 51542 (time 1769759100.15 total 328.55 last 154.12) - open/close 53513 (time 1769759110.16 total 338.56 last 196.97) - open/close 55492 (time 1769759120.16 total 348.56 last 197.89) - open/close 57458 (time 1769759130.17 total 358.56 last 196.47) - open/close 59255 (time 1769759140.17 total 368.57 last 179.68) - open/close 60000 (time 1769759144.05 total 372.45 last 191.77) - open/close 61872 (time 1769759154.06 total 382.45 last 187.13) - open/close 63538 (time 1769759164.06 total 392.46 last 166.55) total: 64768 open/close in 398.97 seconds: 162.34 ops/second - /unlink 3477 (time 1769759182.25 total 10.00 last 347.67) - /unlink 6536 (time 1769759192.26 total 20.01 last 305.76) - /unlink 9775 (time 1769759202.26 total 30.01 last 323.87) - /unlink 13117 (time 1769759212.26 total 40.01 last 334.16) - /unlink 16150 (time 1769759222.26 total 50.01 last 303.21) - /unlink 19362 (time 1769759232.26 total 60.01 last 321.12) - /unlink 20000 (time 1769759235.08 total 62.83 last 226.60) - /unlink 23949 (time 1769759245.08 total 72.83 last 394.80) - /unlink 27953 (time 1769759255.09 total 82.84 last 400.16) - /unlink 30000 (time 1769759260.84 total 88.59 last 356.02) - /unlink 34441 (time 1769759270.84 total 98.59 last 444.03) - /unlink 38666 (time 1769759280.84 total 108.59 last 422.32) - /unlink 40000 (time 1769759283.49 total 111.24 last 504.69) - /unlink 44398 (time 1769759293.49 total 121.24 last 439.64) - /unlink 48799 (time 1769759303.49 total 131.24 last 440.07) - /unlink 50000 (time 1769759306.13 total 133.88 last 455.43) - /unlink 54547 (time 1769759316.13 total 143.88 last 454.68) - /unlink 59158 (time 1769759326.13 total 153.88 last 461.09) - /unlink 63505 (time 1769759336.13 total 163.88 last 434.62) total: 64768 /unlink in 166.52 seconds: 388.94 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg347-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg347-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg347-server stop mds service on oleg347-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg347-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg347-server' oleg347-server: oleg347-server.virtnet: executing unload_modules_local oleg347-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg347-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg347-client: oleg347-client: ssh exited with exit code 2 pdsh@oleg347-client: oleg347-server: ssh exited with exit code 2 pdsh@oleg347-client: oleg347-client: ssh exited with exit code 2 pdsh@oleg347-client: oleg347-server: ssh exited with exit code 2