-----============= acceptance-small: conf-sanity ============----- Thu May 1 21:00:38 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg339-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg339-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg339-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg339-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg339-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg339-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg339-server oleg339-server: oleg339-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg339-server' oleg339-server: oleg339-server.virtnet: executing load_modules_local oleg339-server: Loading modules from /home/green/git/lustre-release/lustre oleg339-server: detected 4 online CPUs by sysfs oleg339-server: MODOPTS_LIBCFS= oleg339-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg339-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg339-server: oleg339-server.virtnet: executing set_default_debug -1 all pdsh@oleg339-client: oleg339-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg339-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg339-server: oleg339-server.virtnet: executing set_default_debug -1 all pdsh@oleg339-client: oleg339-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg339-server: oleg339-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg339-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg339-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg339-server stop mds service on oleg339-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg339-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 21:01:34 (1746147694) Stopping clients: oleg339-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg339-client.virtnet /mnt/lustre2 (opts:-f) oleg339-server: oleg339-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg339-server' oleg339-server: oleg339-server.virtnet: executing load_modules_local oleg339-server: Loading modules from /home/green/git/lustre-release/lustre oleg339-server: detected 4 online CPUs by sysfs oleg339-server: MODOPTS_LIBCFS= oleg339-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg339-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg339-server: oleg339-server.virtnet: executing set_default_debug -1 all pdsh@oleg339-client: oleg339-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg339-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg339-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg339-server: oleg339-server.virtnet: executing set_default_debug -1 all pdsh@oleg339-client: oleg339-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg339-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg339-client.virtnet: -o user_xattr,flock 192.168.203.139@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1452 (time 1746147747.36 total 10.00 last 145.14) - open/close 2950 (time 1746147757.37 total 20.01 last 149.75) - open/close 4525 (time 1746147767.37 total 30.01 last 157.43) - open/close 6446 (time 1746147777.37 total 40.02 last 192.00) - open/close 7807 (time 1746147787.38 total 50.02 last 136.01) - open/close 9100 (time 1746147797.38 total 60.02 last 129.28) - open/close 10000 (time 1746147805.67 total 68.31 last 108.65) - open/close 11070 (time 1746147815.67 total 78.31 last 106.96) - open/close 12397 (time 1746147825.67 total 88.32 last 132.62) - open/close 13578 (time 1746147835.68 total 98.32 last 118.06) - open/close 14912 (time 1746147845.69 total 108.33 last 133.27) - open/close 16136 (time 1746147855.70 total 118.34 last 122.30) - open/close 17393 (time 1746147865.70 total 128.35 last 125.61) - open/close 18661 (time 1746147875.70 total 138.35 last 126.78) - open/close 19850 (time 1746147885.71 total 148.35 last 118.87) - open/close 21018 (time 1746147895.71 total 158.36 last 116.73) - open/close 22362 (time 1746147905.72 total 168.36 last 134.37) - open/close 23731 (time 1746147915.72 total 178.36 last 136.86) - open/close 25079 (time 1746147925.72 total 188.36 last 134.76) - open/close 26534 (time 1746147935.73 total 198.37 last 145.35) - open/close 27986 (time 1746147945.74 total 208.38 last 145.09) - open/close 29249 (time 1746147955.75 total 218.39 last 126.20) - open/close 30000 (time 1746147961.03 total 223.67 last 142.17) - open/close 31503 (time 1746147971.03 total 233.67 last 150.26) - open/close 32635 (time 1746147981.05 total 243.69 last 113.03) - open/close 33897 (time 1746147991.05 total 253.69 last 126.15) - open/close 35145 (time 1746148001.06 total 263.70 last 124.74) - open/close 36552 (time 1746148011.06 total 273.71 last 140.58) - open/close 37671 (time 1746148021.07 total 283.71 last 111.84) - open/close 38889 (time 1746148031.07 total 293.72 last 121.75) - open/close 40000 (time 1746148040.12 total 302.77 last 122.75) - open/close 41402 (time 1746148050.13 total 312.77 last 140.17) - open/close 42746 (time 1746148060.13 total 322.77 last 134.34) - open/close 43917 (time 1746148070.14 total 332.78 last 117.05) - open/close 45015 (time 1746148080.14 total 342.78 last 109.78) - open/close 46175 (time 1746148090.14 total 352.78 last 116.00) - open/close 47248 (time 1746148100.14 total 362.78 last 107.29) - open/close 48445 (time 1746148110.14 total 372.78 last 119.70) - open/close 49709 (time 1746148120.14 total 382.78 last 126.36) - open/close 50000 (time 1746148123.47 total 386.11 last 87.51) - open/close 51102 (time 1746148133.47 total 396.11 last 110.14) - open/close 52194 (time 1746148143.47 total 406.12 last 109.18) - open/close 53543 (time 1746148153.48 total 416.12 last 134.82) - open/close 54963 (time 1746148163.49 total 426.13 last 141.88) - open/close 56377 (time 1746148173.50 total 436.14 last 141.26) - open/close 57790 (time 1746148183.50 total 446.15 last 141.25) - open/close 59014 (time 1746148193.50 total 456.15 last 122.39) - open/close 59994 (time 1746148203.51 total 466.15 last 97.98) - open/close 61254 (time 1746148213.51 total 476.15 last 125.94) - open/close 62547 (time 1746148223.52 total 486.16 last 129.23) - open/close 63530 (time 1746148233.53 total 496.17 last 98.17) - open/close 64538 (time 1746148243.54 total 506.18 last 100.73) total: 64768 open/close in 508.33 seconds: 127.41 ops/second - /unlink 1722 (time 1746148257.47 total 10.00 last 172.14) - /unlink 3489 (time 1746148267.47 total 20.00 last 176.68) - /unlink 5249 (time 1746148277.47 total 30.01 last 175.94) - /unlink 6829 (time 1746148287.47 total 40.01 last 157.95) - /unlink 8390 (time 1746148297.48 total 50.02 last 156.02) - /unlink 9797 (time 1746148307.48 total 60.02 last 140.68) - /unlink 11243 (time 1746148317.49 total 70.02 last 144.53) - /unlink 12997 (time 1746148327.49 total 80.02 last 175.38) - /unlink 14859 (time 1746148337.49 total 90.03 last 186.14) - /unlink 16465 (time 1746148347.49 total 100.03 last 160.55) - /unlink 18236 (time 1746148357.50 total 110.04 last 177.01) - /unlink 20000 (time 1746148366.95 total 119.49 last 186.68) - /unlink 21888 (time 1746148376.95 total 129.49 last 188.73) - /unlink 23745 (time 1746148386.96 total 139.49 last 185.63) - /unlink 25651 (time 1746148396.96 total 149.50 last 190.55) - /unlink 27921 (time 1746148406.96 total 159.50 last 226.90) - /unlink 29806 (time 1746148416.97 total 169.51 last 188.35) - /unlink 31396 (time 1746148426.98 total 179.52 last 158.80) - /unlink 33014 (time 1746148436.98 total 189.52 last 161.78) - /unlink 34618 (time 1746148446.99 total 199.53 last 160.27) - /unlink 36354 (time 1746148456.99 total 209.53 last 173.57) - /unlink 38205 (time 1746148467.00 total 219.53 last 185.07) - /unlink 40000 (time 1746148475.97 total 228.51 last 199.98) - /unlink 41668 (time 1746148485.98 total 238.52 last 166.69) - /unlink 43275 (time 1746148495.99 total 248.52 last 160.59) - /unlink 45399 (time 1746148505.99 total 258.52 last 212.37) - /unlink 47375 (time 1746148515.99 total 268.53 last 197.49) - /unlink 49097 (time 1746148525.99 total 278.53 last 172.16) - /unlink 50000 (time 1746148531.32 total 283.85 last 169.66) - /unlink 51984 (time 1746148541.32 total 293.86 last 198.35) - /unlink 53853 (time 1746148551.33 total 303.86 last 186.78) - /unlink 55422 (time 1746148561.33 total 313.86 last 156.88) - /unlink 57087 (time 1746148571.33 total 323.87 last 166.43) - /unlink 58886 (time 1746148581.33 total 333.87 last 179.88) - /unlink 60000 (time 1746148586.65 total 339.19 last 209.29) - /unlink 61902 (time 1746148596.66 total 349.20 last 190.11) - /unlink 63726 (time 1746148606.66 total 359.20 last 182.32) total: 64768 /unlink in 364.83 seconds: 177.53 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 986 (time 1746148638.92 total 10.00 last 98.56) - open/close 2038 (time 1746148648.92 total 20.01 last 105.17) - open/close 3222 (time 1746148658.92 total 30.01 last 118.36) - open/close 4200 (time 1746148668.93 total 40.02 last 97.71) - open/close 5301 (time 1746148678.94 total 50.02 last 110.07) - open/close 6407 (time 1746148688.94 total 60.02 last 110.60) - open/close 7574 (time 1746148698.94 total 70.03 last 116.66) - open/close 8589 (time 1746148708.95 total 80.04 last 101.40) - open/close 9652 (time 1746148718.96 total 90.04 last 106.24) - open/close 10000 (time 1746148722.50 total 93.58 last 98.28) - open/close 11020 (time 1746148732.51 total 103.59 last 101.89) - open/close 11973 (time 1746148742.53 total 113.61 last 95.13) - open/close 12927 (time 1746148752.53 total 123.62 last 95.32) - open/close 14056 (time 1746148762.53 total 133.62 last 112.89) - open/close 15244 (time 1746148772.54 total 143.62 last 118.79) - open/close 16268 (time 1746148782.55 total 153.64 last 102.25) - open/close 17306 (time 1746148792.55 total 163.64 last 103.79) - open/close 18353 (time 1746148802.56 total 173.64 last 104.64) - open/close 19516 (time 1746148812.56 total 183.64 last 116.29) - open/close 20000 (time 1746148817.69 total 188.77 last 94.37) - open/close 21064 (time 1746148827.69 total 198.78 last 106.36) - open/close 22506 (time 1746148837.69 total 208.78 last 144.16) - open/close 24140 (time 1746148847.69 total 218.78 last 163.37) - open/close 25563 (time 1746148857.70 total 228.79 last 142.18) - open/close 26867 (time 1746148867.71 total 238.80 last 130.32) - open/close 27993 (time 1746148877.73 total 248.82 last 112.35) - open/close 28906 (time 1746148887.73 total 258.82 last 91.30) - open/close 29834 (time 1746148897.74 total 268.82 last 92.74) - open/close 30998 (time 1746148907.74 total 278.83 last 116.33) - open/close 31979 (time 1746148917.75 total 288.83 last 98.09) - open/close 32989 (time 1746148927.75 total 298.83 last 100.98) - open/close 34127 (time 1746148937.75 total 308.83 last 113.78) - open/close 35265 (time 1746148947.75 total 318.84 last 113.72) - open/close 36406 (time 1746148957.76 total 328.85 last 114.02) - open/close 37601 (time 1746148967.76 total 338.85 last 119.50) - open/close 38848 (time 1746148977.77 total 348.85 last 124.66) - open/close 40000 (time 1746148987.21 total 358.30 last 121.97) - open/close 41449 (time 1746148997.21 total 368.30 last 144.89) - open/close 42715 (time 1746149007.21 total 378.30 last 126.60) - open/close 43892 (time 1746149017.21 total 388.30 last 117.69) - open/close 44966 (time 1746149027.22 total 398.30 last 107.33) - open/close 46140 (time 1746149037.22 total 408.31 last 117.33) - open/close 47206 (time 1746149047.23 total 418.31 last 106.58) - open/close 48201 (time 1746149057.23 total 428.31 last 99.48) - open/close 49243 (time 1746149067.23 total 438.32 last 104.13) - open/close 50000 (time 1746149074.61 total 445.69 last 102.67) - open/close 50920 (time 1746149084.62 total 455.71 last 91.85) - open/close 51829 (time 1746149094.65 total 465.73 last 90.68) - open/close 52815 (time 1746149104.68 total 475.77 last 98.28) - open/close 53802 (time 1746149114.69 total 485.78 last 98.57) - open/close 54807 (time 1746149124.71 total 495.79 last 100.38) - open/close 55828 (time 1746149134.71 total 505.80 last 102.02) - open/close 56835 (time 1746149144.73 total 515.82 last 100.50) - open/close 57865 (time 1746149154.74 total 525.82 last 102.96) - open/close 58814 (time 1746149164.74 total 535.83 last 94.84) - open/close 59790 (time 1746149174.75 total 545.84 last 97.49) - open/close 60000 (time 1746149176.89 total 547.98 last 98.18) - open/close 61069 (time 1746149186.90 total 557.98 last 106.88) - open/close 61993 (time 1746149196.90 total 567.99 last 92.35) - open/close 62971 (time 1746149206.91 total 577.99 last 97.75) - open/close 63996 (time 1746149216.91 total 588.00 last 102.46) total: 64768 open/close in 594.73 seconds: 108.90 ops/second - /unlink 1546 (time 1746149235.50 total 10.00 last 154.57) - /unlink 3207 (time 1746149245.51 total 20.01 last 166.00) - /unlink 4912 (time 1746149255.51 total 30.01 last 170.44) - /unlink 6982 (time 1746149265.52 total 40.02 last 206.92) - /unlink 8852 (time 1746149275.52 total 50.02 last 186.91) - /unlink 10000 (time 1746149281.77 total 56.27 last 183.62) - /unlink 11809 (time 1746149291.78 total 66.28 last 180.73) - /unlink 13633 (time 1746149301.79 total 76.28 last 182.35) - /unlink 15701 (time 1746149311.79 total 86.28 last 206.80) - /unlink 17752 (time 1746149321.79 total 96.29 last 205.05) - /unlink 19795 (time 1746149331.79 total 106.29 last 204.21) - /unlink 21638 (time 1746149341.79 total 116.29 last 184.28) - /unlink 23400 (time 1746149351.80 total 126.30 last 176.14) - /unlink 25800 (time 1746149361.80 total 136.30 last 239.93) - /unlink 27806 (time 1746149371.80 total 146.30 last 200.56) - /unlink 29660 (time 1746149381.81 total 156.31 last 185.30) - /unlink 30000 (time 1746149383.92 total 158.42 last 161.03) - /unlink 32129 (time 1746149393.92 total 168.42 last 212.82) - /unlink 33744 (time 1746149403.92 total 178.42 last 161.47) - /unlink 35352 (time 1746149413.93 total 188.43 last 160.76) - /unlink 37249 (time 1746149423.93 total 198.43 last 189.65) - /unlink 38990 (time 1746149433.93 total 208.43 last 174.10) - /unlink 40000 (time 1746149440.52 total 215.01 last 153.33) - /unlink 41566 (time 1746149450.52 total 225.02 last 156.54) - /unlink 43269 (time 1746149460.52 total 235.02 last 170.22) - /unlink 45067 (time 1746149470.53 total 245.02 last 179.77) - /unlink 47015 (time 1746149480.53 total 255.03 last 194.78) - /unlink 48738 (time 1746149490.53 total 265.03 last 172.17) - /unlink 50000 (time 1746149496.65 total 271.15 last 206.33) - /unlink 51750 (time 1746149506.65 total 281.15 last 174.98) - /unlink 53629 (time 1746149516.65 total 291.15 last 187.85) - /unlink 56244 (time 1746149526.65 total 301.15 last 261.49) - /unlink 58151 (time 1746149536.65 total 311.15 last 190.70) - /unlink 60000 (time 1746149546.29 total 320.79 last 191.85) - /unlink 62374 (time 1746149556.29 total 330.79 last 237.39) - /unlink 64268 (time 1746149566.30 total 340.79 last 189.37) total: 64768 /unlink in 343.53 seconds: 188.54 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1010 (time 1746149600.10 total 10.00 last 100.97) - open/close 1990 (time 1746149610.10 total 20.01 last 97.96) - open/close 3117 (time 1746149620.11 total 30.01 last 112.64) - open/close 4154 (time 1746149630.11 total 40.02 last 103.62) - open/close 5139 (time 1746149640.11 total 50.02 last 98.50) - open/close 6116 (time 1746149650.12 total 60.02 last 97.67) - open/close 7197 (time 1746149660.13 total 70.03 last 107.99) - open/close 8246 (time 1746149670.13 total 80.04 last 104.84) - open/close 9236 (time 1746149680.14 total 90.05 last 98.91) - open/close 10000 (time 1746149687.33 total 97.23 last 106.32) - open/close 11185 (time 1746149697.33 total 107.24 last 118.42) - open/close 12148 (time 1746149707.35 total 117.25 last 96.19) - open/close 13088 (time 1746149717.35 total 127.26 last 93.95) - open/close 14256 (time 1746149727.35 total 137.26 last 116.78) - open/close 15333 (time 1746149737.35 total 147.26 last 107.68) - open/close 16385 (time 1746149747.37 total 157.27 last 105.08) - open/close 17530 (time 1746149757.37 total 167.28 last 114.42) - open/close 18733 (time 1746149767.38 total 177.29 last 120.16) - open/close 19719 (time 1746149777.39 total 187.29 last 98.56) - open/close 20000 (time 1746149780.35 total 190.26 last 94.89) - open/close 20982 (time 1746149790.35 total 200.26 last 98.19) - open/close 22370 (time 1746149800.36 total 210.27 last 138.63) - open/close 23580 (time 1746149810.37 total 220.27 last 120.93) - open/close 24680 (time 1746149820.37 total 230.28 last 109.96) - open/close 25849 (time 1746149830.37 total 240.28 last 116.87) - open/close 26808 (time 1746149840.39 total 250.30 last 95.73) - open/close 27824 (time 1746149850.40 total 260.30 last 101.56) - open/close 28967 (time 1746149860.40 total 270.31 last 114.25) - open/close 30000 (time 1746149868.85 total 278.76 last 122.22) - open/close 31079 (time 1746149878.86 total 288.76 last 107.85) - open/close 32217 (time 1746149888.86 total 298.77 last 113.77) - open/close 33707 (time 1746149898.86 total 308.77 last 148.94) - open/close 34968 (time 1746149908.87 total 318.78 last 126.04) - open/close 36296 (time 1746149918.87 total 328.78 last 132.75) - open/close 37486 (time 1746149928.88 total 338.78 last 118.94) - open/close 38526 (time 1746149938.89 total 348.79 last 103.93) - open/close 39584 (time 1746149948.89 total 358.80 last 105.72) - open/close 40000 (time 1746149951.55 total 361.46 last 156.44) - open/close 41177 (time 1746149961.56 total 371.47 last 117.60) - open/close 42130 (time 1746149971.57 total 381.47 last 95.22) - open/close 43288 (time 1746149981.58 total 391.48 last 115.72) - open/close 44425 (time 1746149991.59 total 401.50 last 113.54) - open/close 45399 (time 1746150001.60 total 411.50 last 97.31) - open/close 46516 (time 1746150011.60 total 421.51 last 111.65) - open/close 47506 (time 1746150021.62 total 431.52 last 98.84) - open/close 48421 (time 1746150031.62 total 441.53 last 91.46) - open/close 49246 (time 1746150041.64 total 451.55 last 82.35) - open/close 50000 (time 1746150049.69 total 459.60 last 93.67) - open/close 51114 (time 1746150059.70 total 469.60 last 111.36) - open/close 51928 (time 1746150069.70 total 479.60 last 81.37) - open/close 52847 (time 1746150079.70 total 489.61 last 91.89) - open/close 53803 (time 1746150089.71 total 499.62 last 95.50) - open/close 54729 (time 1746150099.73 total 509.63 last 92.46) - open/close 55545 (time 1746150109.73 total 519.64 last 81.54) - open/close 56434 (time 1746150119.74 total 529.64 last 88.88) - open/close 57320 (time 1746150129.75 total 539.65 last 88.48) - open/close 58373 (time 1746150139.76 total 549.66 last 105.24) - open/close 59349 (time 1746150149.76 total 559.67 last 97.55) - open/close 60000 (time 1746150156.93 total 566.84 last 90.79) - open/close 60911 (time 1746150166.94 total 576.84 last 91.04) - open/close 61975 (time 1746150176.94 total 586.84 last 106.38) - open/close 62942 (time 1746150186.94 total 596.85 last 96.68) - open/close 63860 (time 1746150196.95 total 606.86 last 91.72) total: 64768 open/close in 615.96 seconds: 105.15 ops/second - /unlink 1148 (time 1746150217.85 total 10.00 last 114.74) - /unlink 2036 (time 1746150227.86 total 20.01 last 88.77) - /unlink 2935 (time 1746150237.86 total 30.02 last 89.83) - /unlink 3940 (time 1746150247.87 total 40.02 last 100.44) - /unlink 4927 (time 1746150257.87 total 50.03 last 98.68) - /unlink 5798 (time 1746150267.88 total 60.03 last 87.03) - /unlink 6650 (time 1746150277.88 total 70.04 last 85.17) - /unlink 7572 (time 1746150287.89 total 80.04 last 92.19) - /unlink 8497 (time 1746150297.89 total 90.04 last 92.45) - /unlink 9415 (time 1746150307.90 total 100.05 last 91.75) - /unlink 10000 (time 1746150314.07 total 106.22 last 94.78) - /unlink 11185 (time 1746150324.07 total 116.23 last 118.43) - /unlink 12174 (time 1746150334.09 total 126.24 last 98.79) - /unlink 13141 (time 1746150344.11 total 136.26 last 96.45) - /unlink 14206 (time 1746150354.12 total 146.27 last 106.45) - /unlink 15272 (time 1746150364.12 total 156.27 last 106.60) - /unlink 16149 (time 1746150374.13 total 166.28 last 87.61) - /unlink 17186 (time 1746150384.13 total 176.28 last 103.67) - /unlink 18358 (time 1746150394.14 total 186.29 last 117.09) - /unlink 19221 (time 1746150404.14 total 196.29 last 86.29) - /unlink 20000 (time 1746150412.31 total 204.46 last 95.35) - /unlink 21178 (time 1746150422.31 total 214.47 last 117.77) - /unlink 22195 (time 1746150432.32 total 224.47 last 101.62) - /unlink 23014 (time 1746150442.32 total 234.48 last 81.89) - /unlink 23975 (time 1746150452.32 total 244.48 last 96.09) - /unlink 25052 (time 1746150462.33 total 254.48 last 107.66) - /unlink 25964 (time 1746150472.34 total 264.49 last 91.13) - /unlink 26937 (time 1746150482.34 total 274.49 last 97.25) - /unlink 28012 (time 1746150492.34 total 284.49 last 107.48) - /unlink 29103 (time 1746150502.36 total 294.51 last 108.94) - /unlink 30000 (time 1746150511.01 total 303.16 last 103.68) - /unlink 31237 (time 1746150521.01 total 313.17 last 123.63) - /unlink 32684 (time 1746150531.02 total 323.17 last 144.60) - /unlink 33751 (time 1746150541.03 total 333.18 last 106.64) - /unlink 34756 (time 1746150551.03 total 343.18 last 100.46) - /unlink 35937 (time 1746150561.03 total 353.18 last 118.09) - /unlink 36963 (time 1746150571.04 total 363.19 last 102.52) - /unlink 38070 (time 1746150581.04 total 373.20 last 110.63) - /unlink 39248 (time 1746150591.05 total 383.20 last 117.74) - /unlink 40000 (time 1746150597.57 total 389.72 last 115.29) - /unlink 41036 (time 1746150607.58 total 399.73 last 103.53) - /unlink 42124 (time 1746150617.59 total 409.74 last 108.68) - /unlink 43312 (time 1746150627.59 total 419.75 last 118.75) - /unlink 44347 (time 1746150637.60 total 429.75 last 103.46) - /unlink 45435 (time 1746150647.60 total 439.75 last 108.75) - /unlink 46458 (time 1746150657.61 total 449.76 last 102.26) - /unlink 47531 (time 1746150667.61 total 459.76 last 107.27) - /unlink 48629 (time 1746150677.61 total 469.76 last 109.78) - /unlink 49745 (time 1746150687.62 total 479.77 last 111.54) - /unlink 50000 (time 1746150690.42 total 482.57 last 90.92) - /unlink 51098 (time 1746150700.42 total 492.57 last 109.78) - /unlink 52047 (time 1746150710.43 total 502.58 last 94.82) - /unlink 53095 (time 1746150720.44 total 512.59 last 104.73) - /unlink 54287 (time 1746150730.44 total 522.59 last 119.17) - /unlink 55317 (time 1746150740.45 total 532.60 last 102.89) - /unlink 56424 (time 1746150750.46 total 542.61 last 110.64) - /unlink 57651 (time 1746150760.46 total 552.61 last 122.70) - /unlink 58660 (time 1746150770.46 total 562.62 last 100.83) - /unlink 59744 (time 1746150780.47 total 572.62 last 108.32) - /unlink 60000 (time 1746150782.52 total 574.67 last 124.86) - /unlink 61095 (time 1746150792.53 total 584.68 last 109.42) - /unlink 62078 (time 1746150802.53 total 594.69 last 98.25) - /unlink 63057 (time 1746150812.54 total 604.69 last 97.88) - /unlink 64154 (time 1746150822.54 total 614.69 last 109.68) total: 64768 /unlink in 621.10 seconds: 104.28 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1038 (time 1746150863.76 total 10.01 last 103.70) - open/close 2086 (time 1746150873.76 total 20.01 last 104.77) - open/close 3055 (time 1746150883.76 total 30.02 last 96.86) - open/close 4000 (time 1746150893.78 total 40.03 last 94.35) - open/close 4936 (time 1746150903.79 total 50.04 last 93.53) - open/close 5804 (time 1746150913.81 total 60.06 last 86.64) - open/close 6675 (time 1746150923.81 total 70.06 last 87.09) - open/close 7643 (time 1746150933.82 total 80.07 last 96.72) - open/close 8544 (time 1746150943.82 total 90.07 last 90.07) - open/close 9432 (time 1746150953.82 total 100.08 last 88.76) - open/close 10000 (time 1746150959.53 total 105.78 last 99.61) - open/close 11011 (time 1746150969.54 total 115.79 last 100.94) - open/close 12014 (time 1746150979.55 total 125.80 last 100.27) - open/close 13120 (time 1746150989.55 total 135.80 last 110.57) - open/close 14152 (time 1746150999.55 total 145.81 last 103.15) - open/close 15217 (time 1746151009.56 total 155.81 last 106.42) - open/close 16094 (time 1746151019.57 total 165.82 last 87.64) - open/close 17179 (time 1746151029.57 total 175.82 last 108.47) - open/close 18295 (time 1746151039.58 total 185.83 last 111.55) - open/close 19501 (time 1746151049.59 total 195.84 last 120.44) - open/close 20000 (time 1746151054.42 total 200.68 last 103.23) - open/close 20957 (time 1746151064.43 total 210.68 last 95.62) - open/close 21989 (time 1746151074.43 total 220.69 last 103.18) - open/close 23049 (time 1746151084.44 total 230.69 last 105.95) - open/close 24006 (time 1746151094.45 total 240.70 last 95.63) - open/close 25093 (time 1746151104.45 total 250.71 last 108.61) - open/close 26154 (time 1746151114.46 total 260.71 last 106.07) - open/close 27182 (time 1746151124.46 total 270.71 last 102.78) - open/close 28178 (time 1746151134.46 total 280.71 last 99.56) - open/close 29392 (time 1746151144.47 total 290.72 last 121.29) - open/close 30000 (time 1746151149.10 total 295.35 last 131.37) - open/close 31259 (time 1746151159.10 total 305.35 last 125.87) - open/close 32539 (time 1746151169.11 total 315.36 last 127.94) - open/close 33796 (time 1746151179.11 total 325.36 last 125.67) - open/close 35163 (time 1746151189.11 total 335.36 last 136.65) - open/close 36335 (time 1746151199.11 total 345.36 last 117.19) - open/close 37308 (time 1746151209.11 total 355.37 last 97.29) - open/close 38369 (time 1746151219.12 total 365.37 last 106.06) - open/close 39413 (time 1746151229.12 total 375.37 last 104.36) - open/close 40000 (time 1746151234.40 total 380.65 last 111.18) - open/close 41201 (time 1746151244.41 total 390.66 last 120.04) - open/close 42456 (time 1746151254.41 total 400.66 last 125.43) - open/close 43433 (time 1746151264.42 total 410.67 last 97.59) - open/close 44541 (time 1746151274.43 total 420.68 last 110.77) - open/close 45541 (time 1746151284.43 total 430.68 last 99.93) - open/close 46423 (time 1746151294.44 total 440.69 last 88.12) - open/close 47346 (time 1746151304.44 total 450.70 last 92.27) - open/close 48502 (time 1746151314.44 total 460.70 last 115.60) - open/close 49618 (time 1746151324.45 total 470.70 last 111.59) - open/close 50000 (time 1746151328.02 total 474.27 last 106.76) - open/close 51226 (time 1746151338.03 total 484.28 last 122.49) - open/close 52608 (time 1746151348.04 total 494.29 last 138.16) - open/close 53927 (time 1746151358.04 total 504.29 last 131.89) - open/close 55060 (time 1746151368.04 total 514.29 last 113.28) - open/close 56132 (time 1746151378.04 total 524.29 last 107.19) - open/close 57459 (time 1746151388.04 total 534.29 last 132.69) - open/close 58557 (time 1746151398.04 total 544.29 last 109.75) - open/close 59831 (time 1746151408.04 total 554.30 last 127.38) - open/close 61268 (time 1746151418.05 total 564.30 last 143.64) - open/close 62539 (time 1746151428.05 total 574.30 last 127.07) - open/close 63803 (time 1746151438.05 total 584.31 last 126.35) total: 64768 open/close in 591.15 seconds: 109.56 ops/second - /unlink 1951 (time 1746151456.52 total 10.00 last 195.07) - /unlink 3935 (time 1746151466.53 total 20.01 last 198.31) - /unlink 6158 (time 1746151476.53 total 30.01 last 222.23) - /unlink 8141 (time 1746151486.53 total 40.01 last 198.29) - /unlink 10000 (time 1746151496.49 total 49.97 last 186.64) - /unlink 12026 (time 1746151506.50 total 59.98 last 202.47) - /unlink 14421 (time 1746151516.50 total 69.98 last 239.45) - /unlink 16427 (time 1746151526.51 total 79.99 last 200.31) - /unlink 18249 (time 1746151536.51 total 89.99 last 182.18) - /unlink 20000 (time 1746151542.80 total 96.28 last 278.68) - /unlink 22417 (time 1746151552.80 total 106.28 last 241.68) - /unlink 24551 (time 1746151562.80 total 116.28 last 213.36) - /unlink 27011 (time 1746151572.81 total 126.28 last 245.89) - /unlink 29620 (time 1746151582.81 total 136.29 last 260.78) - /unlink 32307 (time 1746151592.81 total 146.29 last 268.66) - /unlink 34577 (time 1746151602.81 total 156.29 last 226.97) - /unlink 36735 (time 1746151612.81 total 166.29 last 215.75) - /unlink 38872 (time 1746151622.82 total 176.30 last 213.57) - /unlink 40000 (time 1746151629.04 total 182.52 last 181.48) - /unlink 42197 (time 1746151639.04 total 192.52 last 219.66) - /unlink 44443 (time 1746151649.04 total 202.52 last 224.53) - /unlink 47002 (time 1746151659.04 total 212.52 last 255.89) - /unlink 49040 (time 1746151669.05 total 222.53 last 203.68) - /unlink 50000 (time 1746151673.87 total 227.35 last 198.88) - /unlink 52190 (time 1746151683.88 total 237.36 last 218.92) - /unlink 54277 (time 1746151693.88 total 247.36 last 208.60) - /unlink 56076 (time 1746151703.89 total 257.36 last 179.86) - /unlink 58191 (time 1746151713.89 total 267.37 last 211.30) - /unlink 60000 (time 1746151722.17 total 275.65 last 218.66) - /unlink 62028 (time 1746151732.17 total 285.65 last 202.78) - /unlink 64078 (time 1746151742.17 total 295.65 last 204.90) total: 64768 /unlink in 298.21 seconds: 217.19 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1384 (time 1746151773.73 total 10.00 last 138.36) - open/close 2558 (time 1746151783.73 total 20.00 last 117.38) - open/close 3850 (time 1746151793.73 total 30.01 last 129.13) - open/close 5021 (time 1746151803.74 total 40.02 last 117.02) - open/close 6358 (time 1746151813.74 total 50.02 last 133.64) - open/close 7614 (time 1746151823.74 total 60.02 last 125.60) - open/close 8804 (time 1746151833.75 total 70.02 last 118.99) - open/close 10000 (time 1746151842.87 total 79.14 last 131.14) - open/close 11299 (time 1746151852.87 total 89.15 last 129.84) - open/close 12507 (time 1746151862.87 total 99.15 last 120.79) - open/close 13800 (time 1746151872.88 total 109.15 last 129.25) - open/close 14978 (time 1746151882.88 total 119.16 last 117.74) - open/close 16347 (time 1746151892.88 total 129.16 last 136.88) - open/close 17578 (time 1746151902.88 total 139.16 last 123.09) - open/close 18865 (time 1746151912.89 total 149.16 last 128.64) - open/close 20000 (time 1746151921.59 total 157.86 last 130.49) - open/close 21160 (time 1746151931.59 total 167.87 last 115.94) - open/close 22302 (time 1746151941.59 total 177.87 last 114.17) - open/close 23473 (time 1746151951.60 total 187.87 last 117.05) - open/close 24807 (time 1746151961.61 total 197.88 last 133.26) - open/close 26013 (time 1746151971.61 total 207.89 last 120.55) - open/close 27061 (time 1746151981.63 total 217.91 last 104.62) - open/close 28275 (time 1746151991.63 total 227.91 last 121.38) - open/close 29491 (time 1746152001.63 total 237.91 last 121.60) - open/close 30000 (time 1746152006.02 total 242.29 last 116.04) - open/close 31092 (time 1746152016.02 total 252.30 last 109.13) - open/close 32311 (time 1746152026.03 total 262.30 last 121.88) - open/close 33553 (time 1746152036.03 total 272.31 last 124.11) - open/close 34796 (time 1746152046.04 total 282.32 last 124.23) - open/close 36080 (time 1746152056.05 total 292.33 last 128.25) - open/close 37351 (time 1746152066.05 total 302.33 last 127.07) - open/close 38438 (time 1746152076.05 total 312.33 last 108.68) - open/close 39440 (time 1746152086.07 total 322.35 last 100.03) - open/close 40000 (time 1746152091.55 total 327.83 last 102.20) - open/close 41020 (time 1746152101.55 total 337.83 last 101.98) - open/close 41961 (time 1746152111.56 total 347.83 last 94.06) - open/close 43026 (time 1746152121.56 total 357.84 last 106.47) - open/close 44063 (time 1746152131.57 total 367.84 last 103.62) - open/close 45291 (time 1746152141.57 total 377.85 last 122.75) - open/close 46376 (time 1746152151.58 total 387.86 last 108.41) - open/close 47427 (time 1746152161.58 total 397.86 last 105.10) - open/close 48486 (time 1746152171.58 total 407.86 last 105.87) - open/close 49754 (time 1746152181.58 total 417.86 last 126.78) - open/close 50000 (time 1746152183.87 total 420.15 last 107.42) - open/close 51037 (time 1746152193.88 total 430.16 last 103.62) - open/close 52282 (time 1746152203.89 total 440.16 last 124.43) - open/close 53598 (time 1746152213.89 total 450.17 last 131.53) - open/close 54750 (time 1746152223.89 total 460.17 last 115.19) - open/close 55847 (time 1746152233.90 total 470.18 last 109.59) - open/close 57101 (time 1746152243.91 total 480.18 last 125.35) - open/close 58380 (time 1746152253.91 total 490.19 last 127.83) - open/close 59622 (time 1746152263.91 total 500.19 last 124.19) - open/close 60000 (time 1746152267.27 total 503.55 last 112.68) - open/close 61066 (time 1746152277.27 total 513.55 last 106.60) - open/close 62263 (time 1746152287.27 total 523.55 last 119.65) - open/close 63463 (time 1746152297.28 total 533.56 last 119.91) - open/close 64628 (time 1746152307.29 total 543.56 last 116.43) total: 64768 open/close in 544.91 seconds: 118.86 ops/second - /unlink 1620 (time 1746152320.67 total 10.00 last 161.99) - /unlink 3410 (time 1746152330.68 total 20.01 last 178.90) - /unlink 5214 (time 1746152340.68 total 30.01 last 180.35) - /unlink 6797 (time 1746152350.69 total 40.01 last 158.24) - /unlink 8405 (time 1746152360.69 total 50.01 last 160.78) - /unlink 10000 (time 1746152370.12 total 59.44 last 169.12) - /unlink 12035 (time 1746152380.12 total 69.45 last 203.48) - /unlink 13900 (time 1746152390.12 total 79.45 last 186.45) - /unlink 15506 (time 1746152400.13 total 89.46 last 160.44) - /unlink 17044 (time 1746152410.13 total 99.46 last 153.77) - /unlink 18552 (time 1746152420.13 total 109.46 last 150.80) - /unlink 20000 (time 1746152428.43 total 117.75 last 174.60) - /unlink 21815 (time 1746152438.43 total 127.76 last 181.39) - /unlink 23758 (time 1746152448.44 total 137.76 last 194.24) - /unlink 25821 (time 1746152458.44 total 147.77 last 206.21) - /unlink 27837 (time 1746152468.45 total 157.77 last 201.50) - /unlink 29814 (time 1746152478.45 total 167.77 last 197.70) - /unlink 31655 (time 1746152488.45 total 177.78 last 184.05) - /unlink 33566 (time 1746152498.45 total 187.78 last 191.09) - /unlink 35450 (time 1746152508.45 total 197.78 last 188.38) - /unlink 37176 (time 1746152518.45 total 207.78 last 172.59) - /unlink 38867 (time 1746152528.46 total 217.78 last 169.03) - /unlink 40000 (time 1746152535.72 total 225.05 last 155.91) - /unlink 41647 (time 1746152545.73 total 235.06 last 164.57) - /unlink 43275 (time 1746152555.73 total 245.06 last 162.74) - /unlink 45026 (time 1746152565.73 total 255.06 last 175.09) - /unlink 47104 (time 1746152575.74 total 265.06 last 207.73) - /unlink 48917 (time 1746152585.75 total 275.08 last 181.01) - /unlink 50000 (time 1746152591.20 total 280.52 last 198.99) - /unlink 51906 (time 1746152601.20 total 290.52 last 190.59) - /unlink 53882 (time 1746152611.20 total 300.52 last 197.55) - /unlink 55678 (time 1746152621.20 total 310.53 last 179.54) - /unlink 57270 (time 1746152631.21 total 320.54 last 159.08) - /unlink 58816 (time 1746152641.21 total 330.54 last 154.54) - /unlink 60000 (time 1746152647.96 total 337.29 last 175.49) - /unlink 61749 (time 1746152657.96 total 347.29 last 174.86) - /unlink 63571 (time 1746152667.97 total 357.29 last 182.14) total: 64768 /unlink in 362.93 seconds: 178.46 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg339-client.virtnet /mnt/lustre (opts:)