-----============= acceptance-small: conf-sanity ============----- Thu May 1 20:19:11 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg423-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg423-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg423-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg423-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg423-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg423-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg423-server oleg423-server: oleg423-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory loading modules on: 'oleg423-server' oleg423-server: oleg423-server.virtnet: executing load_modules_local oleg423-server: Loading modules from /home/green/git/lustre-release/lustre oleg423-server: detected 4 online CPUs by sysfs oleg423-server: MODOPTS_LIBCFS= oleg423-server: Force libcfs to create 2 CPU partitions oleg423-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg423-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg423-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg423-server: oleg423-server.virtnet: executing set_default_debug -1 all pdsh@oleg423-client: oleg423-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg423-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg423-server: oleg423-server.virtnet: executing set_default_debug -1 all pdsh@oleg423-client: oleg423-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg423-server: oleg423-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg423-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg423-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg423-server stop mds service on oleg423-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg423-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 20:20:14 (1746145214) Stopping clients: oleg423-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg423-client.virtnet /mnt/lustre2 (opts:-f) oleg423-server: oleg423-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg423-server' oleg423-server: oleg423-server.virtnet: executing load_modules_local oleg423-server: Loading modules from /home/green/git/lustre-release/lustre oleg423-server: detected 4 online CPUs by sysfs oleg423-server: MODOPTS_LIBCFS= oleg423-server: Force libcfs to create 2 CPU partitions oleg423-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg423-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg423-server: oleg423-server.virtnet: executing set_default_debug -1 all pdsh@oleg423-client: oleg423-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg423-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg423-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg423-server: oleg423-server.virtnet: executing set_default_debug -1 all pdsh@oleg423-client: oleg423-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg423-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg423-client.virtnet: -o user_xattr,flock 192.168.204.123@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 2123 (time 1746145257.96 total 10.00 last 212.20) - open/close 4009 (time 1746145267.97 total 20.01 last 188.52) - open/close 5978 (time 1746145277.97 total 30.01 last 196.85) - open/close 7533 (time 1746145287.98 total 40.02 last 155.40) - open/close 9150 (time 1746145297.98 total 50.02 last 161.68) - open/close 10000 (time 1746145301.49 total 53.53 last 242.08) - open/close 11538 (time 1746145311.50 total 63.54 last 153.67) - open/close 13422 (time 1746145321.50 total 73.54 last 188.38) - open/close 16158 (time 1746145331.50 total 83.54 last 273.57) - open/close 18573 (time 1746145341.51 total 93.55 last 241.35) - open/close 20000 (time 1746145348.79 total 100.83 last 195.93) - open/close 21971 (time 1746145358.79 total 110.83 last 197.09) - open/close 24280 (time 1746145368.79 total 120.83 last 230.84) - open/close 26003 (time 1746145378.79 total 130.84 last 172.27) - open/close 28069 (time 1746145388.80 total 140.84 last 206.52) - open/close 29706 (time 1746145398.80 total 150.84 last 163.66) - open/close 31258 (time 1746145408.81 total 160.85 last 155.13) - open/close 33108 (time 1746145418.81 total 170.85 last 184.88) - open/close 35814 (time 1746145428.82 total 180.86 last 270.41) - open/close 37705 (time 1746145438.82 total 190.86 last 189.05) - open/close 39907 (time 1746145448.82 total 200.86 last 220.16) - open/close 41603 (time 1746145458.83 total 210.87 last 169.56) - open/close 43953 (time 1746145468.83 total 220.87 last 234.94) - open/close 46416 (time 1746145478.83 total 230.87 last 246.28) - open/close 48646 (time 1746145488.83 total 240.88 last 222.88) - open/close 50000 (time 1746145497.72 total 249.77 last 152.30) - open/close 51580 (time 1746145507.73 total 259.77 last 158.00) - open/close 53675 (time 1746145517.73 total 269.77 last 209.48) - open/close 55447 (time 1746145527.73 total 279.77 last 177.20) - open/close 57111 (time 1746145537.73 total 289.77 last 166.39) - open/close 58938 (time 1746145547.73 total 299.77 last 182.62) - open/close 60000 (time 1746145555.54 total 307.58 last 135.96) - open/close 61052 (time 1746145565.55 total 317.59 last 105.11) - open/close 62120 (time 1746145575.56 total 327.60 last 106.71) - open/close 63156 (time 1746145585.56 total 337.60 last 103.57) - open/close 64420 (time 1746145595.56 total 347.61 last 126.37) total: 64768 open/close in 349.61 seconds: 185.26 ops/second - /unlink 2052 (time 1746145608.81 total 10.00 last 205.15) - /unlink 4388 (time 1746145618.81 total 20.00 last 233.57) - /unlink 6855 (time 1746145628.81 total 30.00 last 246.70) - /unlink 8986 (time 1746145638.81 total 40.00 last 213.08) - /unlink 10000 (time 1746145643.65 total 44.85 last 209.46) - /unlink 12340 (time 1746145653.65 total 54.85 last 233.96) - /unlink 14846 (time 1746145663.66 total 64.86 last 250.38) - /unlink 17039 (time 1746145673.67 total 74.86 last 219.15) - /unlink 19266 (time 1746145683.67 total 84.87 last 222.59) - /unlink 20000 (time 1746145686.45 total 87.65 last 264.22) - /unlink 22352 (time 1746145696.45 total 97.65 last 235.13) - /unlink 24633 (time 1746145706.46 total 107.65 last 228.08) - /unlink 27426 (time 1746145716.46 total 117.65 last 279.24) - /unlink 30000 (time 1746145725.61 total 126.81 last 281.16) - /unlink 32725 (time 1746145735.61 total 136.81 last 272.44) - /unlink 35316 (time 1746145745.61 total 146.81 last 259.09) - /unlink 38586 (time 1746145755.62 total 156.81 last 326.95) - /unlink 40000 (time 1746145760.73 total 161.92 last 276.71) - /unlink 43155 (time 1746145770.73 total 171.92 last 315.48) - /unlink 45400 (time 1746145780.73 total 181.92 last 224.47) - /unlink 48911 (time 1746145790.73 total 191.92 last 351.08) - /unlink 50000 (time 1746145793.25 total 194.45 last 431.52) - /unlink 52680 (time 1746145803.25 total 204.45 last 267.98) - /unlink 55267 (time 1746145813.25 total 214.45 last 258.69) - /unlink 58458 (time 1746145823.25 total 224.45 last 319.08) - /unlink 60000 (time 1746145828.17 total 229.37 last 313.60) - /unlink 62946 (time 1746145838.17 total 239.37 last 294.60) total: 64768 /unlink in 245.08 seconds: 264.27 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1982 (time 1746145870.94 total 10.00 last 198.19) - open/close 3795 (time 1746145880.94 total 20.01 last 181.21) - open/close 5583 (time 1746145890.94 total 30.01 last 178.78) - open/close 7232 (time 1746145900.94 total 40.01 last 164.88) - open/close 8723 (time 1746145910.94 total 50.01 last 149.07) - open/close 9978 (time 1746145920.95 total 60.01 last 125.45) - open/close 11295 (time 1746145930.95 total 70.02 last 131.68) - open/close 12599 (time 1746145940.96 total 80.02 last 130.30) - open/close 14091 (time 1746145950.96 total 90.03 last 149.11) - open/close 15334 (time 1746145960.97 total 100.03 last 124.24) - open/close 16568 (time 1746145970.97 total 110.04 last 123.35) - open/close 17904 (time 1746145980.97 total 120.04 last 133.57) - open/close 19444 (time 1746145990.98 total 130.04 last 153.97) - open/close 20000 (time 1746145995.57 total 134.63 last 121.15) - open/close 21174 (time 1746146005.57 total 144.64 last 117.34) - open/close 22311 (time 1746146015.58 total 154.64 last 113.61) - open/close 23443 (time 1746146025.59 total 164.65 last 113.11) - open/close 24708 (time 1746146035.59 total 174.66 last 126.46) - open/close 26016 (time 1746146045.60 total 184.66 last 130.72) - open/close 27270 (time 1746146055.60 total 194.67 last 125.34) - open/close 28589 (time 1746146065.61 total 204.68 last 131.79) - open/close 29928 (time 1746146075.61 total 214.68 last 133.86) - open/close 31365 (time 1746146085.61 total 224.68 last 143.67) - open/close 32656 (time 1746146095.62 total 234.68 last 129.09) - open/close 34097 (time 1746146105.62 total 244.69 last 144.03) - open/close 35670 (time 1746146115.62 total 254.69 last 157.28) - open/close 36943 (time 1746146125.62 total 264.69 last 127.29) - open/close 38007 (time 1746146135.63 total 274.69 last 106.34) - open/close 39133 (time 1746146145.63 total 284.70 last 112.55) - open/close 40000 (time 1746146152.74 total 291.81 last 122.00) - open/close 41360 (time 1746146162.75 total 301.81 last 135.93) - open/close 43182 (time 1746146172.75 total 311.81 last 182.12) - open/close 44943 (time 1746146182.75 total 321.82 last 176.09) - open/close 46597 (time 1746146192.76 total 331.82 last 165.31) - open/close 48023 (time 1746146202.76 total 341.82 last 142.59) - open/close 49395 (time 1746146212.76 total 351.83 last 137.11) - open/close 50000 (time 1746146216.03 total 355.10 last 185.16) - open/close 51545 (time 1746146226.03 total 365.10 last 154.43) - open/close 52777 (time 1746146236.04 total 375.10 last 123.14) - open/close 54196 (time 1746146246.05 total 385.11 last 141.80) - open/close 55525 (time 1746146256.05 total 395.12 last 132.80) - open/close 56737 (time 1746146266.06 total 405.13 last 121.12) - open/close 57853 (time 1746146276.06 total 415.13 last 111.59) - open/close 59532 (time 1746146286.07 total 425.13 last 167.80) - open/close 60000 (time 1746146289.44 total 428.50 last 138.93) - open/close 61737 (time 1746146299.44 total 438.50 last 173.64) - open/close 63118 (time 1746146309.45 total 448.51 last 137.97) total: 64768 open/close in 458.09 seconds: 141.39 ops/second - /unlink 2066 (time 1746146330.47 total 10.00 last 206.56) - /unlink 3993 (time 1746146340.48 total 20.01 last 192.59) - /unlink 6509 (time 1746146350.48 total 30.01 last 251.49) - /unlink 8851 (time 1746146360.49 total 40.02 last 234.12) - /unlink 10000 (time 1746146366.04 total 45.57 last 206.96) - /unlink 12047 (time 1746146376.04 total 55.57 last 204.56) - /unlink 14147 (time 1746146386.05 total 65.58 last 209.98) - /unlink 16496 (time 1746146396.05 total 75.58 last 234.88) - /unlink 19086 (time 1746146406.05 total 85.58 last 258.82) - /unlink 20000 (time 1746146409.51 total 89.04 last 264.18) - /unlink 22611 (time 1746146419.52 total 99.05 last 261.02) - /unlink 25224 (time 1746146429.52 total 109.05 last 261.29) - /unlink 27328 (time 1746146439.52 total 119.05 last 210.39) - /unlink 29743 (time 1746146449.52 total 129.05 last 241.48) - /unlink 32303 (time 1746146459.52 total 139.05 last 255.94) - /unlink 34656 (time 1746146469.52 total 149.05 last 235.24) - /unlink 37248 (time 1746146479.52 total 159.05 last 259.18) - /unlink 39305 (time 1746146489.53 total 169.06 last 205.63) - /unlink 40000 (time 1746146492.54 total 172.07 last 230.72) - /unlink 42107 (time 1746146502.54 total 182.07 last 210.63) - /unlink 44038 (time 1746146512.55 total 192.08 last 193.00) - /unlink 45846 (time 1746146522.55 total 202.08 last 180.73) - /unlink 47958 (time 1746146532.55 total 212.08 last 211.19) - /unlink 50000 (time 1746146541.84 total 221.37 last 219.89) - /unlink 52520 (time 1746146551.84 total 231.37 last 251.95) - /unlink 55140 (time 1746146561.84 total 241.37 last 261.87) - /unlink 57854 (time 1746146571.85 total 251.38 last 271.34) - /unlink 60000 (time 1746146579.93 total 259.46 last 265.51) - /unlink 63258 (time 1746146589.93 total 269.46 last 325.73) total: 64768 /unlink in 275.51 seconds: 235.08 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1469 (time 1746146622.55 total 10.00 last 146.85) - open/close 3283 (time 1746146632.55 total 20.01 last 181.31) - open/close 4612 (time 1746146642.55 total 30.01 last 132.88) - open/close 6489 (time 1746146652.55 total 40.01 last 187.70) - open/close 8214 (time 1746146662.56 total 50.01 last 172.41) - open/close 9740 (time 1746146672.56 total 60.02 last 152.55) - open/close 10000 (time 1746146674.60 total 62.06 last 127.31) - open/close 11445 (time 1746146684.61 total 72.06 last 144.43) - open/close 12706 (time 1746146694.61 total 82.07 last 126.08) - open/close 14077 (time 1746146704.62 total 92.07 last 136.98) - open/close 15548 (time 1746146714.62 total 102.08 last 147.03) - open/close 17150 (time 1746146724.62 total 112.08 last 160.19) - open/close 18785 (time 1746146734.63 total 122.09 last 163.33) - open/close 20000 (time 1746146743.13 total 130.59 last 142.94) - open/close 21388 (time 1746146753.14 total 140.59 last 138.77) - open/close 22814 (time 1746146763.14 total 150.60 last 142.53) - open/close 24521 (time 1746146773.15 total 160.60 last 170.62) - open/close 26251 (time 1746146783.15 total 170.60 last 172.99) - open/close 27696 (time 1746146793.15 total 180.61 last 144.46) - open/close 29200 (time 1746146803.16 total 190.62 last 150.24) - open/close 30000 (time 1746146810.91 total 198.36 last 103.26) - open/close 31062 (time 1746146820.91 total 208.37 last 106.18) - open/close 32158 (time 1746146830.92 total 218.37 last 109.52) - open/close 33407 (time 1746146840.92 total 228.38 last 124.85) - open/close 34700 (time 1746146850.93 total 238.38 last 129.24) - open/close 36130 (time 1746146860.93 total 248.39 last 142.92) - open/close 37412 (time 1746146870.94 total 258.39 last 128.13) - open/close 38496 (time 1746146880.95 total 268.40 last 108.30) - open/close 39567 (time 1746146890.95 total 278.41 last 107.02) - open/close 40000 (time 1746146894.58 total 282.03 last 119.57) - open/close 41316 (time 1746146904.58 total 292.03 last 131.57) - open/close 42530 (time 1746146914.58 total 302.04 last 121.35) - open/close 43550 (time 1746146924.59 total 312.05 last 101.88) - open/close 44711 (time 1746146934.60 total 322.06 last 115.99) - open/close 45834 (time 1746146944.60 total 332.06 last 112.29) - open/close 47094 (time 1746146954.61 total 342.07 last 125.92) - open/close 48064 (time 1746146964.61 total 352.07 last 96.99) - open/close 49098 (time 1746146974.61 total 362.07 last 103.37) - open/close 50000 (time 1746146983.82 total 371.28 last 97.95) - open/close 51057 (time 1746146993.83 total 381.29 last 105.57) - open/close 52212 (time 1746147003.84 total 391.30 last 115.44) - open/close 53366 (time 1746147013.84 total 401.30 last 115.39) - open/close 54590 (time 1746147023.85 total 411.31 last 122.23) - open/close 55735 (time 1746147033.86 total 421.32 last 114.42) - open/close 57004 (time 1746147043.87 total 431.33 last 126.80) - open/close 58120 (time 1746147053.88 total 441.34 last 111.49) - open/close 59306 (time 1746147063.89 total 451.35 last 118.44) - open/close 60000 (time 1746147069.88 total 457.34 last 115.89) - open/close 61214 (time 1746147079.89 total 467.35 last 121.30) - open/close 62518 (time 1746147089.89 total 477.35 last 130.34) - open/close 63825 (time 1746147099.90 total 487.35 last 130.69) total: 64768 open/close in 496.07 seconds: 130.56 ops/second - /unlink 1911 (time 1746147120.46 total 10.00 last 191.03) - /unlink 4001 (time 1746147130.46 total 20.00 last 208.98) - /unlink 6094 (time 1746147140.47 total 30.01 last 209.17) - /unlink 8122 (time 1746147150.47 total 40.01 last 202.72) - /unlink 10000 (time 1746147158.32 total 47.86 last 239.49) - /unlink 11753 (time 1746147168.32 total 57.86 last 175.19) - /unlink 13577 (time 1746147178.32 total 67.86 last 182.38) - /unlink 15363 (time 1746147188.33 total 77.87 last 178.48) - /unlink 17540 (time 1746147198.33 total 87.87 last 217.68) - /unlink 19563 (time 1746147208.34 total 97.88 last 202.15) - /unlink 20000 (time 1746147210.73 total 100.27 last 182.51) - /unlink 21713 (time 1746147220.74 total 110.28 last 171.23) - /unlink 23763 (time 1746147230.74 total 120.28 last 204.95) - /unlink 25484 (time 1746147240.74 total 130.28 last 172.06) - /unlink 27395 (time 1746147250.74 total 140.28 last 191.05) - /unlink 29327 (time 1746147260.75 total 150.29 last 193.16) - /unlink 30000 (time 1746147263.89 total 153.43 last 214.01) - /unlink 32374 (time 1746147273.89 total 163.43 last 237.37) - /unlink 34526 (time 1746147283.89 total 173.43 last 215.20) - /unlink 36662 (time 1746147293.90 total 183.44 last 213.55) - /unlink 38717 (time 1746147303.90 total 193.44 last 205.48) - /unlink 40000 (time 1746147309.58 total 199.12 last 225.73) - /unlink 42066 (time 1746147319.58 total 209.12 last 206.57) - /unlink 44373 (time 1746147329.58 total 219.12 last 230.68) - /unlink 46368 (time 1746147339.58 total 229.12 last 199.47) - /unlink 48336 (time 1746147349.58 total 239.12 last 196.80) - /unlink 50000 (time 1746147358.37 total 247.91 last 189.37) - /unlink 51910 (time 1746147368.37 total 257.91 last 190.96) - /unlink 53737 (time 1746147378.38 total 267.92 last 182.65) - /unlink 55885 (time 1746147388.38 total 277.92 last 214.72) - /unlink 57768 (time 1746147398.38 total 287.92 last 188.22) - /unlink 59954 (time 1746147408.39 total 297.93 last 218.55) - /unlink 62982 (time 1746147418.39 total 307.93 last 302.63) total: 64768 /unlink in 315.20 seconds: 205.48 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1200 (time 1746147453.27 total 10.00 last 119.94) - open/close 2567 (time 1746147463.27 total 20.01 last 136.69) - open/close 3941 (time 1746147473.27 total 30.01 last 137.40) - open/close 5321 (time 1746147483.27 total 40.01 last 137.94) - open/close 6651 (time 1746147493.28 total 50.02 last 132.93) - open/close 8014 (time 1746147503.28 total 60.02 last 136.29) - open/close 9430 (time 1746147513.28 total 70.02 last 141.56) - open/close 10000 (time 1746147516.67 total 73.41 last 168.20) - open/close 11717 (time 1746147526.67 total 83.41 last 171.68) - open/close 13020 (time 1746147536.67 total 93.41 last 130.26) - open/close 14199 (time 1746147546.67 total 103.41 last 117.89) - open/close 15511 (time 1746147556.68 total 113.41 last 131.17) - open/close 16766 (time 1746147566.68 total 123.42 last 125.47) - open/close 17892 (time 1746147576.68 total 133.42 last 112.58) - open/close 18984 (time 1746147586.69 total 143.42 last 109.15) - open/close 20000 (time 1746147595.11 total 151.84 last 120.67) - open/close 21160 (time 1746147605.11 total 161.85 last 115.98) - open/close 22413 (time 1746147615.11 total 171.85 last 125.30) - open/close 23438 (time 1746147625.12 total 181.86 last 102.40) - open/close 24319 (time 1746147635.12 total 191.86 last 88.05) - open/close 25145 (time 1746147645.13 total 201.86 last 82.58) - open/close 26119 (time 1746147655.13 total 211.87 last 97.33) - open/close 27124 (time 1746147665.14 total 221.88 last 100.40) - open/close 28269 (time 1746147675.14 total 231.88 last 114.49) - open/close 29590 (time 1746147685.15 total 241.89 last 132.01) - open/close 30000 (time 1746147687.67 total 244.41 last 162.78) - open/close 31410 (time 1746147697.67 total 254.41 last 140.99) - open/close 32732 (time 1746147707.68 total 264.42 last 132.03) - open/close 33964 (time 1746147717.69 total 274.43 last 123.07) - open/close 35488 (time 1746147727.69 total 284.43 last 152.40) - open/close 36982 (time 1746147737.70 total 294.44 last 149.32) - open/close 38565 (time 1746147747.71 total 304.44 last 158.20) - open/close 39959 (time 1746147757.71 total 314.44 last 139.39) - open/close 41315 (time 1746147767.71 total 324.45 last 135.59) - open/close 42609 (time 1746147777.71 total 334.45 last 129.37) - open/close 44023 (time 1746147787.71 total 344.45 last 141.34) - open/close 45265 (time 1746147797.71 total 354.45 last 124.19) - open/close 46544 (time 1746147807.72 total 364.46 last 127.77) - open/close 48079 (time 1746147817.73 total 374.47 last 153.46) - open/close 49628 (time 1746147827.73 total 384.47 last 154.84) - open/close 50000 (time 1746147829.75 total 386.49 last 184.39) - open/close 51875 (time 1746147839.75 total 396.49 last 187.49) - open/close 53520 (time 1746147849.75 total 406.49 last 164.41) - open/close 55394 (time 1746147859.76 total 416.50 last 187.28) - open/close 56723 (time 1746147869.77 total 426.51 last 132.83) - open/close 58102 (time 1746147879.77 total 436.51 last 137.82) - open/close 59456 (time 1746147889.77 total 446.51 last 135.38) - open/close 60000 (time 1746147893.71 total 450.45 last 138.33) - open/close 61549 (time 1746147903.71 total 460.45 last 154.86) - open/close 63043 (time 1746147913.71 total 470.45 last 149.40) total: 64768 open/close in 480.36 seconds: 134.83 ops/second - /unlink 1819 (time 1746147935.32 total 10.00 last 181.86) - /unlink 3929 (time 1746147945.32 total 20.00 last 211.00) - /unlink 5986 (time 1746147955.32 total 30.00 last 205.66) - /unlink 8146 (time 1746147965.32 total 40.01 last 215.94) - /unlink 10000 (time 1746147973.24 total 47.93 last 234.05) - /unlink 12702 (time 1746147983.24 total 57.93 last 270.19) - /unlink 14974 (time 1746147993.25 total 67.93 last 227.16) - /unlink 17159 (time 1746148003.25 total 77.93 last 218.41) - /unlink 18967 (time 1746148013.25 total 87.94 last 180.76) - /unlink 20000 (time 1746148017.92 total 92.61 last 221.08) - /unlink 22019 (time 1746148027.93 total 102.61 last 201.85) - /unlink 23890 (time 1746148037.93 total 112.61 last 187.08) - /unlink 25867 (time 1746148047.93 total 122.61 last 197.67) - /unlink 27966 (time 1746148057.93 total 132.62 last 209.85) - /unlink 30000 (time 1746148067.06 total 141.74 last 222.85) - /unlink 31960 (time 1746148077.06 total 151.75 last 195.96) - /unlink 33888 (time 1746148087.06 total 161.75 last 192.74) - /unlink 35966 (time 1746148097.06 total 171.75 last 207.79) - /unlink 37999 (time 1746148107.07 total 181.75 last 203.18) - /unlink 40000 (time 1746148116.85 total 191.54 last 204.57) - /unlink 42516 (time 1746148126.85 total 201.54 last 251.53) - /unlink 45170 (time 1746148136.86 total 211.55 last 265.22) - /unlink 47816 (time 1746148146.87 total 221.55 last 264.49) - /unlink 50000 (time 1746148155.24 total 229.92 last 260.91) - /unlink 53515 (time 1746148165.24 total 239.92 last 351.42) - /unlink 56745 (time 1746148175.24 total 249.93 last 322.90) - /unlink 59871 (time 1746148185.24 total 259.93 last 312.56) - /unlink 63296 (time 1746148195.24 total 269.93 last 342.49) total: 64768 /unlink in 274.52 seconds: 235.93 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1349 (time 1746148227.66 total 10.00 last 134.90) - open/close 2715 (time 1746148237.67 total 20.00 last 136.59) - open/close 3940 (time 1746148247.67 total 30.00 last 122.47) - open/close 5182 (time 1746148257.67 total 40.01 last 124.12) - open/close 6443 (time 1746148267.68 total 50.01 last 126.06) - open/close 7840 (time 1746148277.68 total 60.02 last 139.62) - open/close 9425 (time 1746148287.69 total 70.03 last 158.33) - open/close 10000 (time 1746148291.26 total 73.60 last 161.22) - open/close 11011 (time 1746148301.27 total 83.60 last 101.04) - open/close 12215 (time 1746148311.27 total 93.61 last 120.31) - open/close 13323 (time 1746148321.28 total 103.62 last 110.73) - open/close 14352 (time 1746148331.28 total 113.62 last 102.87) - open/close 15526 (time 1746148341.29 total 123.62 last 117.38) - open/close 16688 (time 1746148351.29 total 133.63 last 116.11) - open/close 17944 (time 1746148361.29 total 143.63 last 125.60) - open/close 19283 (time 1746148371.30 total 153.63 last 133.81) - open/close 20000 (time 1746148377.71 total 160.04 last 111.91) - open/close 21107 (time 1746148387.72 total 170.06 last 110.54) - open/close 22415 (time 1746148397.73 total 180.06 last 130.74) - open/close 23657 (time 1746148407.73 total 190.07 last 124.14) - open/close 24926 (time 1746148417.73 total 200.07 last 126.89) - open/close 26041 (time 1746148427.73 total 210.07 last 111.47) - open/close 27106 (time 1746148437.76 total 220.09 last 106.26) - open/close 28234 (time 1746148447.76 total 230.10 last 112.73) - open/close 29399 (time 1746148457.76 total 240.10 last 116.49) - open/close 30000 (time 1746148463.43 total 245.76 last 106.14) - open/close 31104 (time 1746148473.43 total 255.76 last 110.39) - open/close 32349 (time 1746148483.43 total 265.76 last 124.47) - open/close 34178 (time 1746148493.43 total 275.77 last 182.87) - open/close 35964 (time 1746148503.43 total 285.77 last 178.55) - open/close 38111 (time 1746148513.43 total 295.77 last 214.69) - open/close 39932 (time 1746148523.44 total 305.77 last 182.04) - open/close 41360 (time 1746148533.44 total 315.77 last 142.79) - open/close 42706 (time 1746148543.44 total 325.77 last 134.60) - open/close 44071 (time 1746148553.44 total 335.77 last 136.48) - open/close 45545 (time 1746148563.44 total 345.78 last 147.36) - open/close 46851 (time 1746148573.45 total 355.78 last 130.54) - open/close 48111 (time 1746148583.45 total 365.78 last 125.99) - open/close 49490 (time 1746148593.45 total 375.78 last 137.87) - open/close 50000 (time 1746148597.20 total 379.53 last 136.15) - open/close 51342 (time 1746148607.20 total 389.53 last 134.18) - open/close 52555 (time 1746148617.20 total 399.53 last 121.27) - open/close 53986 (time 1746148627.20 total 409.54 last 143.03) - open/close 55311 (time 1746148637.21 total 419.54 last 132.45) - open/close 56919 (time 1746148647.21 total 429.55 last 160.72) - open/close 58322 (time 1746148657.22 total 439.55 last 140.23) - open/close 59728 (time 1746148667.22 total 449.55 last 140.59) - open/close 60000 (time 1746148669.25 total 451.58 last 133.91) - open/close 61588 (time 1746148679.25 total 461.59 last 158.72) - open/close 63095 (time 1746148689.26 total 471.59 last 150.66) - open/close 64537 (time 1746148699.26 total 481.59 last 144.18) total: 64768 open/close in 483.34 seconds: 134.00 ops/second - /unlink 2625 (time 1746148712.32 total 10.00 last 262.48) - /unlink 5086 (time 1746148722.33 total 20.01 last 245.89) - /unlink 7196 (time 1746148732.33 total 30.01 last 210.98) - /unlink 9245 (time 1746148742.33 total 40.01 last 204.81) - /unlink 10000 (time 1746148745.62 total 43.30 last 229.79) - /unlink 12412 (time 1746148755.62 total 53.30 last 241.18) - /unlink 14765 (time 1746148765.62 total 63.30 last 235.27) - /unlink 16771 (time 1746148775.63 total 73.31 last 200.50) - /unlink 19017 (time 1746148785.63 total 83.31 last 224.57) - /unlink 20000 (time 1746148790.07 total 87.76 last 221.10) - /unlink 22107 (time 1746148800.08 total 97.76 last 210.64) - /unlink 23864 (time 1746148810.08 total 107.76 last 175.63) - /unlink 25781 (time 1746148820.08 total 117.76 last 191.69) - /unlink 27920 (time 1746148830.08 total 127.76 last 213.88) - /unlink 29997 (time 1746148840.08 total 137.76 last 207.69) - /unlink 32080 (time 1746148850.08 total 147.77 last 208.27) - /unlink 34508 (time 1746148860.09 total 157.77 last 242.75) - /unlink 36919 (time 1746148870.09 total 167.77 last 241.09) - /unlink 38808 (time 1746148880.09 total 177.78 last 188.76) - /unlink 40000 (time 1746148886.78 total 184.46 last 178.35) - /unlink 41839 (time 1746148896.78 total 194.47 last 183.76) - /unlink 44175 (time 1746148906.79 total 204.47 last 233.59) - /unlink 46232 (time 1746148916.79 total 214.47 last 205.61) - /unlink 48566 (time 1746148926.79 total 224.47 last 233.36) - /unlink 50000 (time 1746148933.06 total 230.75 last 228.60) - /unlink 52654 (time 1746148943.07 total 240.75 last 265.38) - /unlink 55312 (time 1746148953.07 total 250.75 last 265.75) - /unlink 58096 (time 1746148963.07 total 260.75 last 278.38) - /unlink 60000 (time 1746148969.59 total 267.27 last 292.11) - /unlink 62289 (time 1746148979.59 total 277.27 last 228.85) - /unlink 64669 (time 1746148989.59 total 287.27 last 237.99) total: 64768 /unlink in 287.66 seconds: 225.15 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg423-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg423-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg423-server stop mds service on oleg423-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg423-server unloading modules via unload_modules_local on: 'oleg423-server' oleg423-server: oleg423-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg423-client: oleg423-client: ssh exited with exit code 2 pdsh@oleg423-client: oleg423-server: ssh exited with exit code 2 pdsh@oleg423-client: oleg423-client: ssh exited with exit code 2 pdsh@oleg423-client: oleg423-server: ssh exited with exit code 2 PASS 106 (3815s) Stopping clients: oleg423-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg423-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg423-client: oleg423-server: ssh exited with exit code 2 oleg423-server: oleg423-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg423-server' oleg423-server: oleg423-server.virtnet: executing load_modules_local oleg423-server: Loading modules from /home/green/git/lustre-release/lustre oleg423-server: detected 4 online CPUs by sysfs oleg423-server: MODOPTS_LIBCFS= oleg423-server: Force libcfs to create 2 CPU partitions oleg423-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg423-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3904 sec ========== 21:24:15 (1746149055) === conf-sanity: start cleanup 21:24:16 (1746149056) === === conf-sanity: finish cleanup 21:24:17 (1746149057) ===