-----============= acceptance-small: conf-sanity ============----- Fri Apr 25 18:23:26 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg137-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg137-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg137-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg137-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg137-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg137-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg137-server oleg137-server: oleg137-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory loading modules on: 'oleg137-server' oleg137-server: oleg137-server.virtnet: executing load_modules_local oleg137-server: Loading modules from /home/green/git/lustre-release/lustre oleg137-server: detected 4 online CPUs by sysfs oleg137-server: MODOPTS_LIBCFS= oleg137-server: Force libcfs to create 2 CPU partitions oleg137-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg137-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg137-server: oleg137-server.virtnet: executing set_default_debug -1 all pdsh@oleg137-client: oleg137-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg137-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg137-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg137-server: oleg137-server.virtnet: executing set_default_debug -1 all pdsh@oleg137-client: oleg137-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg137-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg137-server: oleg137-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg137-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg137-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg137-server stop mds service on oleg137-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg137-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 18:25:02 (1745619902) Stopping clients: oleg137-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg137-client.virtnet /mnt/lustre2 (opts:-f) oleg137-server: oleg137-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory loading modules on: 'oleg137-server' oleg137-server: oleg137-server.virtnet: executing load_modules_local oleg137-server: Loading modules from /home/green/git/lustre-release/lustre oleg137-server: detected 4 online CPUs by sysfs oleg137-server: MODOPTS_LIBCFS= oleg137-server: Force libcfs to create 2 CPU partitions oleg137-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg137-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg137-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg137-server: oleg137-server.virtnet: executing set_default_debug -1 all pdsh@oleg137-client: oleg137-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg137-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg137-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg137-server: oleg137-server.virtnet: executing set_default_debug -1 all pdsh@oleg137-client: oleg137-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg137-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg137-client.virtnet: -o user_xattr,flock 192.168.201.137@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1202 (time 1745619975.44 total 10.01 last 120.06) - open/close 2359 (time 1745619985.44 total 20.01 last 115.66) - open/close 3600 (time 1745619995.45 total 30.02 last 124.06) - open/close 4908 (time 1745620005.45 total 40.02 last 130.77) - open/close 6176 (time 1745620015.46 total 50.03 last 126.73) - open/close 7412 (time 1745620025.46 total 60.03 last 123.56) - open/close 8611 (time 1745620035.47 total 70.04 last 119.80) - open/close 9803 (time 1745620045.47 total 80.04 last 119.20) - open/close 10911 (time 1745620055.47 total 90.04 last 110.74) - open/close 12068 (time 1745620065.48 total 100.05 last 115.67) - open/close 13268 (time 1745620075.48 total 110.05 last 119.95) - open/close 14491 (time 1745620085.48 total 120.05 last 122.29) - open/close 15621 (time 1745620095.48 total 130.05 last 112.99) - open/close 16901 (time 1745620105.49 total 140.06 last 127.95) - open/close 18293 (time 1745620115.49 total 150.06 last 139.18) - open/close 19663 (time 1745620125.49 total 160.06 last 136.94) - open/close 20000 (time 1745620128.36 total 162.93 last 117.31) - open/close 21170 (time 1745620138.37 total 172.94 last 116.94) - open/close 22314 (time 1745620148.38 total 182.95 last 114.25) - open/close 23322 (time 1745620158.39 total 192.96 last 100.73) - open/close 24614 (time 1745620168.39 total 202.96 last 129.18) - open/close 26262 (time 1745620178.39 total 212.96 last 164.80) - open/close 27733 (time 1745620188.39 total 222.96 last 147.08) - open/close 29098 (time 1745620198.39 total 232.96 last 136.48) - open/close 30000 (time 1745620204.59 total 239.16 last 145.51) - open/close 31391 (time 1745620214.59 total 249.16 last 139.09) - open/close 32784 (time 1745620224.60 total 259.17 last 139.16) - open/close 33986 (time 1745620234.61 total 269.18 last 120.16) - open/close 35332 (time 1745620244.61 total 279.18 last 134.50) - open/close 36536 (time 1745620254.62 total 289.19 last 120.35) - open/close 37921 (time 1745620264.62 total 299.19 last 138.43) - open/close 39542 (time 1745620274.63 total 309.20 last 162.01) - open/close 40000 (time 1745620277.42 total 311.99 last 164.19) - open/close 41783 (time 1745620287.42 total 321.99 last 178.23) - open/close 43293 (time 1745620297.42 total 331.99 last 150.98) - open/close 44983 (time 1745620307.43 total 342.00 last 168.93) - open/close 46522 (time 1745620317.44 total 352.01 last 153.75) - open/close 47952 (time 1745620327.44 total 362.01 last 142.99) - open/close 49739 (time 1745620337.44 total 372.01 last 178.60) - open/close 51336 (time 1745620347.45 total 382.02 last 159.57) - open/close 52891 (time 1745620357.45 total 392.02 last 155.48) - open/close 54353 (time 1745620367.46 total 402.03 last 146.08) - open/close 55943 (time 1745620377.47 total 412.04 last 158.94) - open/close 57395 (time 1745620387.47 total 422.04 last 145.20) - open/close 58851 (time 1745620397.47 total 432.04 last 145.59) - open/close 60000 (time 1745620404.63 total 439.20 last 160.41) - open/close 61549 (time 1745620414.63 total 449.20 last 154.83) - open/close 63129 (time 1745620424.63 total 459.20 last 158.00) total: 64768 open/close in 468.41 seconds: 138.27 ops/second - /unlink 3340 (time 1745620445.44 total 10.00 last 333.94) - /unlink 6161 (time 1745620455.44 total 20.00 last 282.08) - /unlink 8957 (time 1745620465.44 total 30.00 last 279.58) - /unlink 10000 (time 1745620469.49 total 34.05 last 257.77) - /unlink 12335 (time 1745620479.50 total 44.06 last 233.30) - /unlink 14696 (time 1745620489.50 total 54.06 last 236.03) - /unlink 17181 (time 1745620499.50 total 64.06 last 248.49) - /unlink 19952 (time 1745620509.50 total 74.06 last 277.05) - /unlink 22375 (time 1745620519.50 total 84.07 last 242.25) - /unlink 24551 (time 1745620529.51 total 94.07 last 217.53) - /unlink 27179 (time 1745620539.51 total 104.07 last 262.79) - /unlink 29550 (time 1745620549.52 total 114.08 last 236.91) - /unlink 30000 (time 1745620554.23 total 118.79 last 95.38) - /unlink 31073 (time 1745620564.24 total 128.80 last 107.28) - /unlink 32090 (time 1745620574.25 total 138.81 last 101.60) - /unlink 33404 (time 1745620584.25 total 148.81 last 131.39) - /unlink 35009 (time 1745620594.25 total 158.81 last 160.48) - /unlink 36592 (time 1745620604.25 total 168.81 last 158.29) - /unlink 38139 (time 1745620614.25 total 178.81 last 154.69) - /unlink 39622 (time 1745620624.26 total 188.82 last 148.20) - /unlink 40000 (time 1745620626.53 total 191.09 last 166.01) - /unlink 41385 (time 1745620636.54 total 201.10 last 138.38) - /unlink 42674 (time 1745620646.54 total 211.10 last 128.90) - /unlink 43939 (time 1745620656.55 total 221.11 last 126.44) - /unlink 45371 (time 1745620666.55 total 231.11 last 143.14) - /unlink 46751 (time 1745620676.55 total 241.11 last 137.98) - /unlink 48190 (time 1745620686.56 total 251.12 last 143.77) - /unlink 49899 (time 1745620696.56 total 261.12 last 170.89) - /unlink 51712 (time 1745620706.56 total 271.12 last 181.29) - /unlink 53507 (time 1745620716.56 total 281.12 last 179.49) - /unlink 55023 (time 1745620726.57 total 291.13 last 151.55) - /unlink 56707 (time 1745620736.57 total 301.13 last 168.35) - /unlink 58247 (time 1745620746.57 total 311.14 last 153.93) - /unlink 60000 (time 1745620756.30 total 320.86 last 180.23) - /unlink 61469 (time 1745620766.30 total 330.86 last 146.89) - /unlink 62897 (time 1745620776.31 total 340.87 last 142.75) - /unlink 64299 (time 1745620786.31 total 350.88 last 140.08) total: 64768 /unlink in 354.37 seconds: 182.77 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 882 (time 1745620820.40 total 10.00 last 88.19) - open/close 1887 (time 1745620830.41 total 20.00 last 100.49) - open/close 2877 (time 1745620840.41 total 30.01 last 98.94) - open/close 3801 (time 1745620850.42 total 40.02 last 92.32) - open/close 4712 (time 1745620860.42 total 50.02 last 91.10) - open/close 5548 (time 1745620870.44 total 60.03 last 83.47) - open/close 6354 (time 1745620880.45 total 70.04 last 80.53) - open/close 7142 (time 1745620890.45 total 80.05 last 78.73) - open/close 8017 (time 1745620900.46 total 90.06 last 87.42) - open/close 8954 (time 1745620910.47 total 100.06 last 93.68) - open/close 9928 (time 1745620920.47 total 110.06 last 97.38) - open/close 10870 (time 1745620930.47 total 120.07 last 94.17) - open/close 11896 (time 1745620940.47 total 130.07 last 102.59) - open/close 12791 (time 1745620950.47 total 140.07 last 89.49) - open/close 13765 (time 1745620960.48 total 150.07 last 97.39) - open/close 14611 (time 1745620970.49 total 160.09 last 84.47) - open/close 15437 (time 1745620980.49 total 170.09 last 82.60) - open/close 16291 (time 1745620990.50 total 180.10 last 85.28) - open/close 17177 (time 1745621000.51 total 190.11 last 88.51) - open/close 17913 (time 1745621010.52 total 200.12 last 73.53) - open/close 18807 (time 1745621020.53 total 210.12 last 89.39) - open/close 19709 (time 1745621030.53 total 220.12 last 90.18) - open/close 20000 (time 1745621034.23 total 223.82 last 78.67) - open/close 20858 (time 1745621044.23 total 233.83 last 85.76) - open/close 21770 (time 1745621054.24 total 243.83 last 91.16) - open/close 22606 (time 1745621064.24 total 253.84 last 83.57) - open/close 23467 (time 1745621074.25 total 263.85 last 86.01) - open/close 24319 (time 1745621084.25 total 273.85 last 85.18) - open/close 25175 (time 1745621094.26 total 283.85 last 85.55) - open/close 26061 (time 1745621104.26 total 293.86 last 88.57) - open/close 26937 (time 1745621114.27 total 303.87 last 87.51) - open/close 27810 (time 1745621124.28 total 313.87 last 87.25) - open/close 28681 (time 1745621134.28 total 323.88 last 87.08) - open/close 29535 (time 1745621144.29 total 333.88 last 85.33) - open/close 30000 (time 1745621150.39 total 339.99 last 76.15) - open/close 30855 (time 1745621160.40 total 350.00 last 85.43) - open/close 31734 (time 1745621170.40 total 360.00 last 87.88) - open/close 32574 (time 1745621180.42 total 370.02 last 83.86) - open/close 33392 (time 1745621190.43 total 380.03 last 81.72) - open/close 34197 (time 1745621200.43 total 390.03 last 80.49) - open/close 35024 (time 1745621210.44 total 400.03 last 82.65) - open/close 35885 (time 1745621220.44 total 410.03 last 86.10) - open/close 36767 (time 1745621230.44 total 420.04 last 88.13) - open/close 37681 (time 1745621240.45 total 430.04 last 91.37) - open/close 38668 (time 1745621250.45 total 440.04 last 98.69) - open/close 39576 (time 1745621260.45 total 450.05 last 90.78) - open/close 40000 (time 1745621264.95 total 454.55 last 94.16) - open/close 41031 (time 1745621274.96 total 464.55 last 103.06) - open/close 42003 (time 1745621284.96 total 474.56 last 97.15) - open/close 43018 (time 1745621294.97 total 484.56 last 101.46) - open/close 43877 (time 1745621304.97 total 494.56 last 85.90) - open/close 44750 (time 1745621314.97 total 504.57 last 87.26) - open/close 45594 (time 1745621324.98 total 514.57 last 84.36) - open/close 46536 (time 1745621334.98 total 524.58 last 94.19) - open/close 47476 (time 1745621344.98 total 534.58 last 93.96) - open/close 48435 (time 1745621354.99 total 544.59 last 95.84) - open/close 49320 (time 1745621364.99 total 554.59 last 88.46) - open/close 50000 (time 1745621372.05 total 561.65 last 96.33) - open/close 50983 (time 1745621382.07 total 571.66 last 98.18) - open/close 51922 (time 1745621392.07 total 581.66 last 93.90) - open/close 52904 (time 1745621402.07 total 591.67 last 98.14) - open/close 53786 (time 1745621412.08 total 601.68 last 88.11) - open/close 54669 (time 1745621422.08 total 611.68 last 88.29) - open/close 55615 (time 1745621432.09 total 621.68 last 94.57) - open/close 56625 (time 1745621442.09 total 631.68 last 100.98) - open/close 57724 (time 1745621452.09 total 641.69 last 109.88) - open/close 58685 (time 1745621462.10 total 651.70 last 96.02) - open/close 59687 (time 1745621472.10 total 661.70 last 100.17) - open/close 60000 (time 1745621475.26 total 664.86 last 99.08) - open/close 61000 (time 1745621485.29 total 674.88 last 99.74) - open/close 61938 (time 1745621495.29 total 684.89 last 93.78) - open/close 62843 (time 1745621505.30 total 694.89 last 90.43) - open/close 63836 (time 1745621515.30 total 704.90 last 99.23) - open/close 64736 (time 1745621525.31 total 714.91 last 89.90) total: 64768 open/close in 715.24 seconds: 90.55 ops/second - /unlink 1012 (time 1745621538.49 total 10.01 last 101.11) - /unlink 1849 (time 1745621548.49 total 20.01 last 83.70) - /unlink 2823 (time 1745621558.52 total 30.04 last 97.09) - /unlink 3784 (time 1745621568.53 total 40.05 last 96.00) - /unlink 4640 (time 1745621578.53 total 50.05 last 85.58) - /unlink 5503 (time 1745621588.53 total 60.06 last 86.26) - /unlink 6379 (time 1745621598.54 total 70.06 last 87.59) - /unlink 7344 (time 1745621608.54 total 80.06 last 96.49) - /unlink 8240 (time 1745621618.55 total 90.08 last 89.45) - /unlink 9225 (time 1745621628.55 total 100.08 last 98.50) - /unlink 10000 (time 1745621636.00 total 107.52 last 104.07) - /unlink 11073 (time 1745621646.00 total 117.53 last 107.27) - /unlink 12062 (time 1745621656.01 total 127.53 last 98.88) - /unlink 12977 (time 1745621666.01 total 137.53 last 91.46) - /unlink 13952 (time 1745621676.01 total 147.54 last 97.49) - /unlink 14850 (time 1745621686.02 total 157.54 last 89.77) - /unlink 15731 (time 1745621696.02 total 167.54 last 88.07) - /unlink 16822 (time 1745621706.02 total 177.55 last 109.06) - /unlink 17711 (time 1745621716.03 total 187.55 last 88.85) - /unlink 18549 (time 1745621726.04 total 197.56 last 83.75) - /unlink 19489 (time 1745621736.05 total 207.57 last 93.88) - /unlink 20000 (time 1745621741.84 total 213.37 last 88.20) - /unlink 20855 (time 1745621751.85 total 223.37 last 85.45) - /unlink 21723 (time 1745621761.86 total 233.38 last 86.72) - /unlink 22628 (time 1745621771.86 total 243.38 last 90.48) - /unlink 23619 (time 1745621781.87 total 253.39 last 98.98) - /unlink 24610 (time 1745621791.89 total 263.41 last 98.90) - /unlink 25457 (time 1745621801.90 total 273.42 last 84.66) - /unlink 26345 (time 1745621811.90 total 283.42 last 88.76) - /unlink 27334 (time 1745621821.91 total 293.43 last 98.85) - /unlink 28365 (time 1745621831.91 total 303.43 last 103.10) - /unlink 29408 (time 1745621841.92 total 313.44 last 104.16) - /unlink 30000 (time 1745621848.25 total 319.77 last 93.54) - /unlink 31079 (time 1745621858.26 total 329.78 last 107.81) - /unlink 32098 (time 1745621868.26 total 339.78 last 101.84) - /unlink 33178 (time 1745621878.27 total 349.79 last 107.95) - /unlink 34238 (time 1745621888.27 total 359.79 last 105.95) - /unlink 35335 (time 1745621898.27 total 369.80 last 109.68) - /unlink 36391 (time 1745621908.27 total 379.80 last 105.60) - /unlink 37404 (time 1745621918.29 total 389.81 last 101.15) - /unlink 38470 (time 1745621928.29 total 399.81 last 106.58) - /unlink 39563 (time 1745621938.31 total 409.83 last 109.12) - /unlink 40000 (time 1745621942.81 total 414.34 last 96.96) - /unlink 41127 (time 1745621952.82 total 424.34 last 112.66) - /unlink 42190 (time 1745621962.82 total 434.35 last 106.24) - /unlink 43269 (time 1745621972.84 total 444.36 last 107.75) - /unlink 44446 (time 1745621982.84 total 454.37 last 117.63) - /unlink 45582 (time 1745621992.84 total 464.37 last 113.58) - /unlink 46904 (time 1745622002.85 total 474.37 last 132.18) - /unlink 48063 (time 1745622012.85 total 484.37 last 115.86) - /unlink 49114 (time 1745622022.86 total 494.38 last 105.03) - /unlink 50000 (time 1745622031.46 total 502.99 last 102.92) - /unlink 51142 (time 1745622041.47 total 512.99 last 114.19) - /unlink 52357 (time 1745622051.48 total 523.00 last 121.36) - /unlink 53613 (time 1745622061.48 total 533.00 last 125.56) - /unlink 54797 (time 1745622071.49 total 543.01 last 118.32) - /unlink 55992 (time 1745622081.49 total 553.01 last 119.50) - /unlink 57253 (time 1745622091.49 total 563.02 last 126.03) - /unlink 58580 (time 1745622101.49 total 573.02 last 132.70) - /unlink 59760 (time 1745622111.50 total 583.02 last 117.94) - /unlink 60000 (time 1745622113.84 total 585.37 last 102.38) - /unlink 61219 (time 1745622123.84 total 595.37 last 121.89) - /unlink 62480 (time 1745622133.84 total 605.37 last 126.08) - /unlink 63681 (time 1745622143.86 total 615.38 last 119.96) - /unlink 64745 (time 1745622153.86 total 625.38 last 106.34) total: 64768 /unlink in 625.59 seconds: 103.53 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1439 (time 1745622182.46 total 10.01 last 143.80) - open/close 2847 (time 1745622192.46 total 20.01 last 140.78) - open/close 4200 (time 1745622202.47 total 30.02 last 135.19) - open/close 5436 (time 1745622212.48 total 40.03 last 123.47) - open/close 6738 (time 1745622222.49 total 50.03 last 130.14) - open/close 8034 (time 1745622232.49 total 60.04 last 129.52) - open/close 9359 (time 1745622242.50 total 70.05 last 132.38) - open/close 10000 (time 1745622248.05 total 75.60 last 115.47) - open/close 11476 (time 1745622258.06 total 85.60 last 147.55) - open/close 12947 (time 1745622268.06 total 95.60 last 147.07) - open/close 14388 (time 1745622278.06 total 105.61 last 144.03) - open/close 15675 (time 1745622288.07 total 115.61 last 128.66) - open/close 16953 (time 1745622298.07 total 125.61 last 127.75) - open/close 18078 (time 1745622308.07 total 135.62 last 112.48) - open/close 19305 (time 1745622318.08 total 145.63 last 122.57) - open/close 20000 (time 1745622324.12 total 151.67 last 115.11) - open/close 21160 (time 1745622334.13 total 161.67 last 115.94) - open/close 22337 (time 1745622344.13 total 171.67 last 117.69) - open/close 23665 (time 1745622354.13 total 181.68 last 132.75) - open/close 24893 (time 1745622364.14 total 191.68 last 122.73) - open/close 26174 (time 1745622374.14 total 201.69 last 128.02) - open/close 27406 (time 1745622384.14 total 211.69 last 123.16) - open/close 28711 (time 1745622394.15 total 221.70 last 130.41) - open/close 30000 (time 1745622403.64 total 231.18 last 135.90) - open/close 31309 (time 1745622413.64 total 241.19 last 130.83) - open/close 32528 (time 1745622423.65 total 251.19 last 121.84) - open/close 33793 (time 1745622433.65 total 261.20 last 126.39) - open/close 34865 (time 1745622443.66 total 271.20 last 107.17) - open/close 35876 (time 1745622453.66 total 281.21 last 101.08) - open/close 37110 (time 1745622463.66 total 291.21 last 123.34) - open/close 38379 (time 1745622473.67 total 301.21 last 126.87) - open/close 39664 (time 1745622483.67 total 311.22 last 128.47) - open/close 40000 (time 1745622485.91 total 313.46 last 149.89) - open/close 41405 (time 1745622495.92 total 323.47 last 140.37) - open/close 42866 (time 1745622505.92 total 333.47 last 146.06) - open/close 44389 (time 1745622515.92 total 343.47 last 152.29) - open/close 45846 (time 1745622525.93 total 353.47 last 145.67) - open/close 47125 (time 1745622535.94 total 363.48 last 127.79) - open/close 48505 (time 1745622545.94 total 373.49 last 137.94) - open/close 49835 (time 1745622555.95 total 383.49 last 132.93) - open/close 51130 (time 1745622565.95 total 393.50 last 129.43) - open/close 52284 (time 1745622575.95 total 403.50 last 115.36) - open/close 53614 (time 1745622585.96 total 413.50 last 132.93) - open/close 54776 (time 1745622595.96 total 423.51 last 116.18) - open/close 55949 (time 1745622605.97 total 433.51 last 117.25) - open/close 57130 (time 1745622615.98 total 443.52 last 117.95) - open/close 58195 (time 1745622625.98 total 453.53 last 106.44) - open/close 59281 (time 1745622635.99 total 463.53 last 108.58) - open/close 60000 (time 1745622642.98 total 470.52 last 102.84) - open/close 60934 (time 1745622652.98 total 480.52 last 93.39) - open/close 61852 (time 1745622662.99 total 490.53 last 91.73) - open/close 62905 (time 1745622672.99 total 500.54 last 105.22) - open/close 64149 (time 1745622682.99 total 510.54 last 124.39) total: 64768 open/close in 516.03 seconds: 125.51 ops/second - /unlink 1695 (time 1745622700.71 total 10.00 last 169.42) - /unlink 3429 (time 1745622710.71 total 20.01 last 173.33) - /unlink 5287 (time 1745622720.72 total 30.01 last 185.73) - /unlink 7258 (time 1745622730.72 total 40.01 last 197.08) - /unlink 9166 (time 1745622740.72 total 50.01 last 190.79) - /unlink 10000 (time 1745622745.51 total 54.81 last 174.09) - /unlink 11596 (time 1745622755.52 total 64.82 last 159.42) - /unlink 13407 (time 1745622765.52 total 74.82 last 181.10) - /unlink 15339 (time 1745622775.52 total 84.82 last 193.16) - /unlink 17303 (time 1745622785.52 total 94.82 last 196.40) - /unlink 19373 (time 1745622795.53 total 104.82 last 206.94) - /unlink 20000 (time 1745622799.00 total 108.30 last 180.48) - /unlink 21831 (time 1745622809.00 total 118.30 last 183.08) - /unlink 23830 (time 1745622819.00 total 128.30 last 199.90) - /unlink 25693 (time 1745622829.00 total 138.30 last 186.27) - /unlink 27510 (time 1745622839.00 total 148.30 last 181.69) - /unlink 29160 (time 1745622849.01 total 158.30 last 164.94) - /unlink 30000 (time 1745622853.78 total 163.07 last 176.08) - /unlink 31650 (time 1745622863.78 total 173.08 last 164.96) - /unlink 33452 (time 1745622873.79 total 183.08 last 180.08) - /unlink 35237 (time 1745622883.79 total 193.09 last 178.39) - /unlink 37150 (time 1745622893.80 total 203.09 last 191.22) - /unlink 39218 (time 1745622903.80 total 213.10 last 206.75) - /unlink 40000 (time 1745622907.45 total 216.75 last 214.12) - /unlink 41803 (time 1745622917.45 total 226.75 last 180.29) - /unlink 43728 (time 1745622927.45 total 236.75 last 192.45) - /unlink 45542 (time 1745622937.46 total 246.76 last 181.30) - /unlink 47370 (time 1745622947.46 total 256.76 last 182.79) - /unlink 49300 (time 1745622957.46 total 266.76 last 192.97) - /unlink 50000 (time 1745622961.54 total 270.83 last 171.72) - /unlink 51779 (time 1745622971.54 total 280.84 last 177.83) - /unlink 53468 (time 1745622981.55 total 290.84 last 168.85) - /unlink 55310 (time 1745622991.55 total 300.84 last 184.16) - /unlink 57080 (time 1745623001.55 total 310.85 last 176.93) - /unlink 59051 (time 1745623011.55 total 320.85 last 197.08) - /unlink 60000 (time 1745623015.68 total 324.97 last 230.08) - /unlink 62240 (time 1745623025.68 total 334.98 last 223.93) - /unlink 64456 (time 1745623035.68 total 344.98 last 221.60) total: 64768 /unlink in 346.41 seconds: 186.97 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1548 (time 1745623068.13 total 10.01 last 154.70) - open/close 3176 (time 1745623078.13 total 20.01 last 162.76) - open/close 4957 (time 1745623088.13 total 30.01 last 178.06) - open/close 6427 (time 1745623098.13 total 40.01 last 146.95) - open/close 7916 (time 1745623108.14 total 50.02 last 148.85) - open/close 9300 (time 1745623118.14 total 60.02 last 138.37) - open/close 10000 (time 1745623122.78 total 64.66 last 150.89) - open/close 11541 (time 1745623132.78 total 74.66 last 154.04) - open/close 12978 (time 1745623142.78 total 84.67 last 143.66) - open/close 14404 (time 1745623152.79 total 94.67 last 142.53) - open/close 15897 (time 1745623162.79 total 104.67 last 149.27) - open/close 17348 (time 1745623172.79 total 114.67 last 145.10) - open/close 18873 (time 1745623182.79 total 124.68 last 152.47) - open/close 20000 (time 1745623189.85 total 131.73 last 159.66) - open/close 21771 (time 1745623199.86 total 141.74 last 177.06) - open/close 23399 (time 1745623209.86 total 151.74 last 162.76) - open/close 25352 (time 1745623219.86 total 161.74 last 195.19) - open/close 27198 (time 1745623229.87 total 171.75 last 184.54) - open/close 28914 (time 1745623239.87 total 181.75 last 171.49) - open/close 30000 (time 1745623245.84 total 187.72 last 182.05) - open/close 32070 (time 1745623255.84 total 197.72 last 206.98) - open/close 33807 (time 1745623265.85 total 207.73 last 173.58) - open/close 35250 (time 1745623275.85 total 217.73 last 144.29) - open/close 36631 (time 1745623285.85 total 227.74 last 138.00) - open/close 38274 (time 1745623295.86 total 237.74 last 164.23) - open/close 39906 (time 1745623305.86 total 247.74 last 163.16) - open/close 41794 (time 1745623315.86 total 257.74 last 188.79) - open/close 43346 (time 1745623325.87 total 267.75 last 155.14) - open/close 45063 (time 1745623335.87 total 277.75 last 171.69) - open/close 46965 (time 1745623345.87 total 287.75 last 190.20) - open/close 48738 (time 1745623355.87 total 297.75 last 177.24) - open/close 50000 (time 1745623362.47 total 304.35 last 191.25) - open/close 52181 (time 1745623372.47 total 314.35 last 218.07) - open/close 54657 (time 1745623382.47 total 324.35 last 247.51) - open/close 56743 (time 1745623392.47 total 334.35 last 208.59) - open/close 58608 (time 1745623402.48 total 344.36 last 186.46) - open/close 60000 (time 1745623411.15 total 353.03 last 160.49) - open/close 61561 (time 1745623421.15 total 363.03 last 156.09) - open/close 63157 (time 1745623431.16 total 373.04 last 159.47) - open/close 64747 (time 1745623441.16 total 383.04 last 158.95) total: 64768 open/close in 383.18 seconds: 169.03 ops/second - /unlink 2962 (time 1745623452.98 total 10.00 last 296.17) - /unlink 6109 (time 1745623462.99 total 20.01 last 314.53) - /unlink 8852 (time 1745623472.99 total 30.01 last 274.26) - /unlink 10000 (time 1745623477.15 total 34.16 last 276.18) - /unlink 13253 (time 1745623487.15 total 44.17 last 325.17) - /unlink 16892 (time 1745623497.15 total 54.17 last 363.87) - /unlink 19923 (time 1745623507.15 total 64.17 last 303.04) - /unlink 24000 (time 1745623517.16 total 74.17 last 407.63) - /unlink 26823 (time 1745623527.16 total 84.18 last 282.13) - /unlink 29953 (time 1745623537.16 total 94.18 last 312.96) - /unlink 33495 (time 1745623547.16 total 104.18 last 354.19) - /unlink 36382 (time 1745623557.16 total 114.18 last 288.67) - /unlink 39510 (time 1745623567.17 total 124.19 last 312.67) - /unlink 42570 (time 1745623577.17 total 134.19 last 306.00) - /unlink 45290 (time 1745623587.17 total 144.19 last 272.00) - /unlink 47341 (time 1745623597.17 total 154.19 last 205.05) - /unlink 49491 (time 1745623607.17 total 164.19 last 214.99) - /unlink 50000 (time 1745623609.53 total 166.54 last 216.03) - /unlink 52801 (time 1745623619.53 total 176.55 last 280.08) - /unlink 55667 (time 1745623629.53 total 186.55 last 286.60) - /unlink 58442 (time 1745623639.53 total 196.55 last 277.44) - /unlink 60000 (time 1745623645.73 total 202.75 last 251.38) - /unlink 62524 (time 1745623655.73 total 212.75 last 252.36) total: 64768 /unlink in 220.46 seconds: 293.79 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1409 (time 1745623690.05 total 10.01 last 140.81) - open/close 2947 (time 1745623700.05 total 20.01 last 153.72) - open/close 4547 (time 1745623710.06 total 30.02 last 159.88) - open/close 6319 (time 1745623720.06 total 40.02 last 177.15) - open/close 7975 (time 1745623730.06 total 50.02 last 165.60) - open/close 9677 (time 1745623740.06 total 60.02 last 170.18) - open/close 11288 (time 1745623750.07 total 70.03 last 160.99) - open/close 13071 (time 1745623760.08 total 80.04 last 178.18) - open/close 15201 (time 1745623770.08 total 90.04 last 212.88) - open/close 17035 (time 1745623780.09 total 100.05 last 183.18) - open/close 18569 (time 1745623790.10 total 110.06 last 153.32) - open/close 20000 (time 1745623798.74 total 118.70 last 165.57) - open/close 21938 (time 1745623808.75 total 128.71 last 193.69) - open/close 23586 (time 1745623818.75 total 138.71 last 164.74) - open/close 25549 (time 1745623828.75 total 148.71 last 196.24) - open/close 27360 (time 1745623838.76 total 158.72 last 180.96) - open/close 29230 (time 1745623848.76 total 168.72 last 186.95) - open/close 30000 (time 1745623852.11 total 172.07 last 230.15) - open/close 32013 (time 1745623862.12 total 182.08 last 201.16) - open/close 33993 (time 1745623872.12 total 192.08 last 198.00) - open/close 36139 (time 1745623882.12 total 202.08 last 214.56) - open/close 38036 (time 1745623892.12 total 212.08 last 189.67) - open/close 40000 (time 1745623901.94 total 221.90 last 200.10) - open/close 42009 (time 1745623911.94 total 231.90 last 200.81) - open/close 43834 (time 1745623921.94 total 241.90 last 182.44) - open/close 45532 (time 1745623931.95 total 251.91 last 169.70) - open/close 47500 (time 1745623941.95 total 261.91 last 196.72) - open/close 49183 (time 1745623951.96 total 271.92 last 168.22) - open/close 50000 (time 1745623956.28 total 276.24 last 188.92) - open/close 52262 (time 1745623966.28 total 286.24 last 226.17) - open/close 54109 (time 1745623976.29 total 296.25 last 184.66) - open/close 56023 (time 1745623986.29 total 306.25 last 191.36) - open/close 58146 (time 1745623996.29 total 316.25 last 212.25) - open/close 60000 (time 1745624005.49 total 325.45 last 201.60) - open/close 62191 (time 1745624015.49 total 335.45 last 219.09) - open/close 64175 (time 1745624025.49 total 345.45 last 198.39) total: 64768 open/close in 348.75 seconds: 185.71 ops/second - /unlink 2884 (time 1745624040.19 total 10.00 last 288.39) - /unlink 6104 (time 1745624050.19 total 20.00 last 321.94) - /unlink 9400 (time 1745624060.19 total 30.01 last 329.46) - /unlink 10000 (time 1745624062.71 total 32.53 last 238.09) - /unlink 13557 (time 1745624072.71 total 42.53 last 355.69) - /unlink 17114 (time 1745624082.71 total 52.53 last 355.63) - /unlink 20000 (time 1745624091.40 total 61.21 last 332.26) - /unlink 22851 (time 1745624101.40 total 71.22 last 285.06) - /unlink 26533 (time 1745624111.40 total 81.22 last 368.17) - /unlink 29876 (time 1745624121.40 total 91.22 last 334.29) - /unlink 32540 (time 1745624131.40 total 101.22 last 266.38) - /unlink 36071 (time 1745624141.41 total 111.22 last 353.05) - /unlink 39633 (time 1745624151.41 total 121.22 last 356.13) - /unlink 43558 (time 1745624161.41 total 131.22 last 392.39) - /unlink 46811 (time 1745624171.41 total 141.22 last 325.30) - /unlink 49835 (time 1745624181.41 total 151.23 last 302.34) - /unlink 52603 (time 1745624191.41 total 161.23 last 276.73) - /unlink 55622 (time 1745624201.42 total 171.23 last 301.79) - /unlink 58876 (time 1745624211.42 total 181.23 last 325.37) - /unlink 60000 (time 1745624215.45 total 185.27 last 278.53) - /unlink 63571 (time 1745624225.46 total 195.27 last 357.03) total: 64768 /unlink in 198.90 seconds: 325.63 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg137-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg137-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg137-server stop mds service on oleg137-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg137-server unloading modules via unload_modules_local on: 'oleg137-server' oleg137-server: oleg137-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg137-client: oleg137-client: ssh exited with exit code 2 pdsh@oleg137-client: oleg137-server: ssh exited with exit code 2 pdsh@oleg137-client: oleg137-client: ssh exited with exit code 2 pdsh@oleg137-client: oleg137-server: ssh exited with exit code 2 PASS 106 (4355s) Stopping clients: oleg137-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg137-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg137-client: oleg137-server: ssh exited with exit code 2 oleg137-server: oleg137-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg137-server' oleg137-server: oleg137-server.virtnet: executing load_modules_local oleg137-server: Loading modules from /home/green/git/lustre-release/lustre oleg137-server: detected 4 online CPUs by sysfs oleg137-server: MODOPTS_LIBCFS= oleg137-server: Force libcfs to create 2 CPU partitions oleg137-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg137-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg137-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 4476 sec ========== 19:38:03 (1745624283) === conf-sanity: start cleanup 19:38:03 (1745624283) === === conf-sanity: finish cleanup 19:38:04 (1745624284) ===