-----============= acceptance-small: conf-sanity ============----- Thu May 1 16:32:38 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg438-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg438-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg438-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg438-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg438-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg438-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg438-server oleg438-server: oleg438-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg438-server' oleg438-server: oleg438-server.virtnet: executing load_modules_local oleg438-server: Loading modules from /home/green/git/lustre-release/lustre oleg438-server: detected 4 online CPUs by sysfs oleg438-server: MODOPTS_LIBCFS= oleg438-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg438-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg438-server: oleg438-server.virtnet: executing set_default_debug -1 all pdsh@oleg438-client: oleg438-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg438-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg438-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg438-server: oleg438-server.virtnet: executing set_default_debug -1 all pdsh@oleg438-client: oleg438-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg438-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg438-server: oleg438-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg438-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg438-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg438-server stop mds service on oleg438-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg438-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 16:33:53 (1746131633) Stopping clients: oleg438-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg438-client.virtnet /mnt/lustre2 (opts:-f) oleg438-server: oleg438-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg438-server' oleg438-server: oleg438-server.virtnet: executing load_modules_local oleg438-server: Loading modules from /home/green/git/lustre-release/lustre oleg438-server: detected 4 online CPUs by sysfs oleg438-server: MODOPTS_LIBCFS= oleg438-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg438-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg438-server: oleg438-server.virtnet: executing set_default_debug -1 all pdsh@oleg438-client: oleg438-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg438-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg438-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg438-server: oleg438-server.virtnet: executing set_default_debug -1 all pdsh@oleg438-client: oleg438-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg438-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg438-client.virtnet: -o user_xattr,flock 192.168.204.138@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 1230 (time 1746131683.87 total 10.00 last 122.99) - open/close 2492 (time 1746131693.87 total 20.00 last 126.20) - open/close 3715 (time 1746131703.88 total 30.01 last 122.15) - open/close 4818 (time 1746131713.89 total 40.02 last 110.23) - open/close 5970 (time 1746131723.89 total 50.02 last 115.16) - open/close 7295 (time 1746131733.89 total 60.03 last 132.44) - open/close 8431 (time 1746131743.90 total 70.03 last 113.52) - open/close 9452 (time 1746131753.91 total 80.04 last 102.05) - open/close 10000 (time 1746131758.73 total 84.86 last 113.70) - open/close 11228 (time 1746131768.73 total 94.87 last 122.71) - open/close 12387 (time 1746131778.74 total 104.87 last 115.82) - open/close 13390 (time 1746131788.74 total 114.87 last 100.29) - open/close 14537 (time 1746131798.74 total 124.88 last 114.68) - open/close 15529 (time 1746131808.74 total 134.88 last 99.19) - open/close 16607 (time 1746131818.75 total 144.88 last 107.78) - open/close 17841 (time 1746131828.75 total 154.88 last 123.34) - open/close 18913 (time 1746131838.76 total 164.89 last 107.11) - open/close 19937 (time 1746131848.77 total 174.90 last 102.35) - open/close 21011 (time 1746131858.77 total 184.90 last 107.38) - open/close 22467 (time 1746131868.77 total 194.90 last 145.56) - open/close 23929 (time 1746131878.78 total 204.91 last 146.11) - open/close 25175 (time 1746131888.79 total 214.92 last 124.49) - open/close 26190 (time 1746131898.79 total 224.92 last 101.44) - open/close 27149 (time 1746131908.80 total 234.93 last 95.83) - open/close 28349 (time 1746131918.80 total 244.94 last 119.96) - open/close 29642 (time 1746131928.80 total 254.94 last 129.28) - open/close 30000 (time 1746131932.51 total 258.64 last 96.63) - open/close 31117 (time 1746131942.52 total 268.66 last 111.52) - open/close 32426 (time 1746131952.53 total 278.66 last 130.82) - open/close 33608 (time 1746131962.53 total 288.67 last 118.14) - open/close 35026 (time 1746131972.54 total 298.67 last 141.79) - open/close 36401 (time 1746131982.54 total 308.67 last 137.44) - open/close 37772 (time 1746131992.54 total 318.67 last 137.08) - open/close 38807 (time 1746132002.55 total 328.68 last 103.46) - open/close 39757 (time 1746132012.55 total 338.68 last 94.99) - open/close 40000 (time 1746132014.71 total 340.85 last 112.12) - open/close 41120 (time 1746132024.72 total 350.85 last 111.96) - open/close 42323 (time 1746132034.72 total 360.85 last 120.25) - open/close 43562 (time 1746132044.72 total 370.86 last 123.88) - open/close 44822 (time 1746132054.72 total 380.86 last 125.99) - open/close 46302 (time 1746132064.73 total 390.86 last 147.97) - open/close 47558 (time 1746132074.73 total 400.86 last 125.57) - open/close 48698 (time 1746132084.73 total 410.87 last 113.94) - open/close 49733 (time 1746132094.74 total 420.87 last 103.47) - open/close 50000 (time 1746132097.54 total 423.67 last 95.31) - open/close 51080 (time 1746132107.54 total 433.67 last 107.99) - open/close 52139 (time 1746132117.54 total 443.67 last 105.90) - open/close 53232 (time 1746132127.54 total 453.68 last 109.27) - open/close 54455 (time 1746132137.55 total 463.68 last 122.22) - open/close 55596 (time 1746132147.56 total 473.69 last 113.99) - open/close 56821 (time 1746132157.57 total 483.70 last 122.35) - open/close 58018 (time 1746132167.57 total 493.71 last 119.67) - open/close 59215 (time 1746132177.58 total 503.71 last 119.66) - open/close 60000 (time 1746132183.36 total 509.50 last 135.65) - open/close 61038 (time 1746132193.37 total 519.50 last 103.75) - open/close 61977 (time 1746132203.37 total 529.50 last 93.86) - open/close 62977 (time 1746132213.37 total 539.50 last 100.00) - open/close 64146 (time 1746132223.38 total 549.51 last 116.82) total: 64768 open/close in 555.12 seconds: 116.67 ops/second - /unlink 1687 (time 1746132240.76 total 10.00 last 168.69) - /unlink 3428 (time 1746132250.76 total 20.00 last 174.10) - /unlink 5127 (time 1746132260.77 total 30.00 last 169.87) - /unlink 6819 (time 1746132270.77 total 40.01 last 169.08) - /unlink 8622 (time 1746132280.78 total 50.02 last 180.19) - /unlink 10000 (time 1746132288.45 total 57.69 last 179.54) - /unlink 11757 (time 1746132298.45 total 67.69 last 175.67) - /unlink 13214 (time 1746132308.46 total 77.70 last 145.60) - /unlink 14742 (time 1746132318.47 total 87.71 last 152.66) - /unlink 16317 (time 1746132328.47 total 97.71 last 157.46) - /unlink 17861 (time 1746132338.48 total 107.71 last 154.35) - /unlink 19295 (time 1746132348.48 total 117.72 last 143.30) - /unlink 20000 (time 1746132353.34 total 122.58 last 145.22) - /unlink 21524 (time 1746132363.34 total 132.58 last 152.34) - /unlink 23450 (time 1746132373.34 total 142.58 last 192.56) - /unlink 25354 (time 1746132383.35 total 152.58 last 190.34) - /unlink 27313 (time 1746132393.35 total 162.59 last 195.88) - /unlink 29359 (time 1746132403.35 total 172.59 last 204.53) - /unlink 30000 (time 1746132406.27 total 175.51 last 219.51) - /unlink 32088 (time 1746132416.27 total 185.51 last 208.75) - /unlink 33974 (time 1746132426.28 total 195.51 last 188.53) - /unlink 36069 (time 1746132436.28 total 205.52 last 209.47) - /unlink 38722 (time 1746132446.28 total 215.52 last 265.28) - /unlink 40000 (time 1746132451.21 total 220.44 last 259.36) - /unlink 42536 (time 1746132461.21 total 230.45 last 253.54) - /unlink 44856 (time 1746132471.21 total 240.45 last 231.94) - /unlink 47146 (time 1746132481.21 total 250.45 last 228.99) - /unlink 49448 (time 1746132491.21 total 260.45 last 230.18) - /unlink 50000 (time 1746132493.53 total 262.77 last 238.03) - /unlink 52070 (time 1746132503.53 total 272.77 last 206.97) - /unlink 54135 (time 1746132513.53 total 282.77 last 206.49) - /unlink 56276 (time 1746132523.54 total 292.77 last 214.08) - /unlink 58564 (time 1746132533.54 total 302.78 last 228.75) - /unlink 60000 (time 1746132541.25 total 310.49 last 186.16) - /unlink 61862 (time 1746132551.25 total 320.49 last 186.15) - /unlink 64113 (time 1746132561.26 total 330.50 last 224.95) total: 64768 /unlink in 334.20 seconds: 193.80 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1312 (time 1746132595.65 total 10.01 last 131.10) - open/close 2545 (time 1746132605.66 total 20.01 last 123.22) - open/close 3763 (time 1746132615.66 total 30.01 last 121.79) - open/close 5047 (time 1746132625.67 total 40.03 last 128.24) - open/close 6532 (time 1746132635.67 total 50.03 last 148.43) - open/close 7923 (time 1746132645.68 total 60.04 last 139.03) - open/close 9246 (time 1746132655.68 total 70.04 last 132.27) - open/close 10000 (time 1746132661.33 total 75.68 last 133.56) - open/close 11288 (time 1746132671.33 total 85.69 last 128.71) - open/close 12720 (time 1746132681.35 total 95.71 last 142.90) - open/close 14234 (time 1746132691.36 total 105.71 last 151.36) - open/close 15799 (time 1746132701.36 total 115.72 last 156.42) - open/close 17169 (time 1746132711.37 total 125.73 last 136.92) - open/close 18444 (time 1746132721.38 total 135.74 last 127.32) - open/close 19806 (time 1746132731.39 total 145.74 last 136.14) - open/close 21057 (time 1746132741.40 total 155.75 last 124.98) - open/close 22482 (time 1746132751.40 total 165.76 last 142.45) - open/close 23937 (time 1746132761.41 total 175.77 last 145.38) - open/close 25330 (time 1746132771.41 total 185.77 last 139.24) - open/close 26929 (time 1746132781.42 total 195.78 last 159.81) - open/close 28349 (time 1746132791.42 total 205.78 last 141.91) - open/close 29907 (time 1746132801.43 total 215.79 last 155.74) - open/close 31553 (time 1746132811.43 total 225.79 last 164.48) - open/close 33056 (time 1746132821.44 total 235.79 last 150.27) - open/close 34798 (time 1746132831.44 total 245.80 last 174.15) - open/close 36352 (time 1746132841.44 total 255.80 last 155.33) - open/close 37936 (time 1746132851.45 total 265.81 last 158.32) - open/close 39324 (time 1746132861.45 total 275.81 last 138.77) - open/close 40000 (time 1746132866.14 total 280.50 last 144.13) - open/close 41224 (time 1746132876.14 total 290.50 last 122.40) - open/close 42413 (time 1746132886.14 total 300.50 last 118.89) - open/close 43576 (time 1746132896.14 total 310.50 last 116.27) - open/close 44816 (time 1746132906.16 total 320.52 last 123.82) - open/close 46065 (time 1746132916.17 total 330.53 last 124.75) - open/close 47500 (time 1746132926.17 total 340.53 last 143.46) - open/close 49132 (time 1746132936.17 total 350.53 last 163.19) - open/close 50000 (time 1746132941.32 total 355.68 last 168.53) - open/close 51691 (time 1746132951.33 total 365.69 last 169.04) - open/close 53018 (time 1746132961.33 total 375.69 last 132.69) - open/close 54219 (time 1746132971.33 total 385.69 last 120.09) - open/close 55411 (time 1746132981.34 total 395.70 last 119.07) - open/close 56535 (time 1746132991.34 total 405.70 last 112.38) - open/close 57793 (time 1746133001.34 total 415.70 last 125.78) - open/close 59286 (time 1746133011.35 total 425.71 last 149.20) - open/close 60000 (time 1746133016.41 total 430.77 last 141.04) - open/close 61261 (time 1746133026.43 total 440.78 last 125.94) - open/close 62378 (time 1746133036.45 total 450.81 last 111.45) - open/close 63484 (time 1746133046.45 total 460.81 last 110.59) - open/close 64675 (time 1746133056.46 total 470.81 last 119.03) total: 64768 open/close in 471.59 seconds: 137.34 ops/second - /unlink 1963 (time 1746133069.05 total 10.00 last 196.24) - /unlink 3660 (time 1746133079.06 total 20.01 last 169.64) - /unlink 5480 (time 1746133089.06 total 30.01 last 181.98) - /unlink 7602 (time 1746133099.06 total 40.01 last 212.16) - /unlink 9924 (time 1746133109.06 total 50.01 last 232.15) - /unlink 12167 (time 1746133119.07 total 60.01 last 224.24) - /unlink 14207 (time 1746133129.07 total 70.01 last 203.99) - /unlink 16664 (time 1746133139.07 total 80.02 last 245.53) - /unlink 18754 (time 1746133149.08 total 90.02 last 208.94) - /unlink 20000 (time 1746133155.33 total 96.28 last 199.29) - /unlink 22308 (time 1746133165.33 total 106.28 last 230.74) - /unlink 24041 (time 1746133175.33 total 116.28 last 173.29) - /unlink 25774 (time 1746133185.33 total 126.28 last 173.29) - /unlink 27387 (time 1746133195.34 total 136.29 last 161.16) - /unlink 28997 (time 1746133205.34 total 146.29 last 160.99) - /unlink 30000 (time 1746133211.37 total 152.32 last 166.46) - /unlink 31609 (time 1746133221.37 total 162.32 last 160.89) - /unlink 33523 (time 1746133231.37 total 172.32 last 191.34) - /unlink 35194 (time 1746133241.38 total 182.32 last 167.03) - /unlink 37182 (time 1746133251.38 total 192.33 last 198.66) - /unlink 39089 (time 1746133261.38 total 202.33 last 190.65) - /unlink 40000 (time 1746133266.60 total 207.55 last 174.74) - /unlink 42106 (time 1746133276.60 total 217.55 last 210.56) - /unlink 44004 (time 1746133286.60 total 227.55 last 189.78) - /unlink 45979 (time 1746133296.61 total 237.56 last 197.39) - /unlink 47794 (time 1746133306.61 total 247.56 last 181.45) - /unlink 49691 (time 1746133316.61 total 257.56 last 189.66) - /unlink 51761 (time 1746133326.61 total 267.56 last 206.99) - /unlink 53773 (time 1746133336.61 total 277.56 last 201.18) - /unlink 55978 (time 1746133346.62 total 287.57 last 220.42) - /unlink 58378 (time 1746133356.62 total 297.57 last 239.98) - /unlink 60000 (time 1746133362.93 total 303.88 last 256.87) - /unlink 62301 (time 1746133372.94 total 313.89 last 229.95) - /unlink 64061 (time 1746133382.95 total 323.89 last 175.89) total: 64768 /unlink in 328.22 seconds: 197.33 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1228 (time 1746133415.19 total 10.01 last 122.73) - open/close 2384 (time 1746133425.19 total 20.01 last 115.57) - open/close 3775 (time 1746133435.20 total 30.01 last 139.05) - open/close 5076 (time 1746133445.21 total 40.02 last 129.96) - open/close 6395 (time 1746133455.21 total 50.03 last 131.87) - open/close 7813 (time 1746133465.21 total 60.03 last 141.79) - open/close 8994 (time 1746133475.21 total 70.03 last 118.08) - open/close 10000 (time 1746133482.71 total 77.53 last 134.08) - open/close 11184 (time 1746133492.72 total 87.54 last 118.33) - open/close 12508 (time 1746133502.72 total 97.54 last 132.35) - open/close 13696 (time 1746133512.73 total 107.55 last 118.69) - open/close 14866 (time 1746133522.74 total 117.56 last 116.91) - open/close 16325 (time 1746133532.74 total 127.56 last 145.86) - open/close 17700 (time 1746133542.75 total 137.57 last 137.42) - open/close 19224 (time 1746133552.75 total 147.57 last 152.35) - open/close 20000 (time 1746133556.79 total 151.61 last 192.10) - open/close 21360 (time 1746133566.79 total 161.61 last 135.99) - open/close 22677 (time 1746133576.80 total 171.61 last 131.67) - open/close 23885 (time 1746133586.80 total 181.62 last 120.74) - open/close 25157 (time 1746133596.80 total 191.62 last 127.20) - open/close 26954 (time 1746133606.80 total 201.62 last 179.62) - open/close 28457 (time 1746133616.81 total 211.63 last 150.23) - open/close 30000 (time 1746133626.41 total 221.22 last 160.75) - open/close 31878 (time 1746133636.41 total 231.23 last 187.77) - open/close 33811 (time 1746133646.41 total 241.23 last 193.28) - open/close 35793 (time 1746133656.41 total 251.23 last 198.18) - open/close 37124 (time 1746133666.41 total 261.23 last 133.08) - open/close 38355 (time 1746133676.42 total 271.23 last 123.03) - open/close 39899 (time 1746133686.42 total 281.24 last 154.36) - open/close 41628 (time 1746133696.43 total 291.24 last 172.81) - open/close 43107 (time 1746133706.43 total 301.25 last 147.84) - open/close 44667 (time 1746133716.43 total 311.25 last 155.95) - open/close 46482 (time 1746133726.43 total 321.25 last 181.48) - open/close 48863 (time 1746133736.43 total 331.25 last 238.10) - open/close 50000 (time 1746133742.90 total 337.72 last 175.70) - open/close 51880 (time 1746133752.91 total 347.72 last 187.94) - open/close 53924 (time 1746133762.91 total 357.73 last 204.26) - open/close 55980 (time 1746133772.92 total 367.73 last 205.55) - open/close 57763 (time 1746133782.92 total 377.73 last 178.30) - open/close 60000 (time 1746133792.84 total 387.66 last 225.45) - open/close 62377 (time 1746133802.84 total 397.66 last 237.67) total: 64768 open/close in 407.31 seconds: 159.01 ops/second - /unlink 4321 (time 1746133823.45 total 10.00 last 432.03) - /unlink 8011 (time 1746133833.46 total 20.00 last 368.97) - /unlink 10000 (time 1746133839.45 total 25.99 last 331.99) - /unlink 14711 (time 1746133849.45 total 36.00 last 470.97) - /unlink 19171 (time 1746133859.45 total 46.00 last 445.95) - /unlink 20000 (time 1746133861.55 total 48.10 last 394.73) - /unlink 23176 (time 1746133871.55 total 58.10 last 317.51) - /unlink 26422 (time 1746133881.55 total 68.10 last 324.58) - /unlink 28793 (time 1746133891.56 total 78.10 last 237.02) - /unlink 30000 (time 1746133894.92 total 81.47 last 358.92) - /unlink 33435 (time 1746133904.92 total 91.47 last 343.41) - /unlink 36285 (time 1746133914.93 total 101.47 last 284.88) - /unlink 38742 (time 1746133924.93 total 111.48 last 245.66) - /unlink 40000 (time 1746133930.58 total 117.13 last 222.69) - /unlink 42693 (time 1746133940.58 total 127.13 last 269.27) - /unlink 46170 (time 1746133950.58 total 137.13 last 347.66) - /unlink 50000 (time 1746133957.67 total 144.22 last 540.37) - /unlink 55298 (time 1746133967.67 total 154.22 last 529.72) - /unlink 60000 (time 1746133977.26 total 163.81 last 490.17) - /unlink 64018 (time 1746133987.26 total 173.81 last 401.74) total: 64768 /unlink in 176.21 seconds: 367.56 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1617 (time 1746134017.49 total 10.00 last 161.66) - open/close 3334 (time 1746134027.49 total 20.01 last 171.60) - open/close 5046 (time 1746134037.50 total 30.02 last 171.07) - open/close 6382 (time 1746134047.50 total 40.02 last 133.60) - open/close 8340 (time 1746134057.50 total 50.02 last 195.73) - open/close 10000 (time 1746134066.72 total 59.24 last 180.09) - open/close 11899 (time 1746134076.73 total 69.24 last 189.82) - open/close 13904 (time 1746134086.73 total 79.25 last 200.41) - open/close 15919 (time 1746134096.73 total 89.25 last 201.49) - open/close 18476 (time 1746134106.73 total 99.25 last 255.66) - open/close 20000 (time 1746134116.14 total 108.66 last 161.96) - open/close 21556 (time 1746134126.16 total 118.67 last 155.36) - open/close 22983 (time 1746134136.16 total 128.68 last 142.66) - open/close 24368 (time 1746134146.17 total 138.68 last 138.42) - open/close 25891 (time 1746134156.17 total 148.68 last 152.28) - open/close 27207 (time 1746134166.17 total 158.69 last 131.56) - open/close 28818 (time 1746134176.17 total 168.69 last 161.08) - open/close 30000 (time 1746134183.37 total 175.89 last 164.16) - open/close 31977 (time 1746134193.38 total 185.89 last 197.64) - open/close 34047 (time 1746134203.38 total 195.89 last 206.94) - open/close 35432 (time 1746134213.38 total 205.90 last 138.49) - open/close 36676 (time 1746134223.39 total 215.91 last 124.23) - open/close 37935 (time 1746134233.40 total 225.91 last 125.86) - open/close 39595 (time 1746134243.40 total 235.92 last 165.94) - open/close 40000 (time 1746134245.57 total 238.08 last 186.82) - open/close 42174 (time 1746134255.57 total 248.09 last 217.29) - open/close 43922 (time 1746134265.58 total 258.09 last 174.74) - open/close 45642 (time 1746134275.58 total 268.09 last 171.96) - open/close 47416 (time 1746134285.58 total 278.10 last 177.33) - open/close 49733 (time 1746134295.58 total 288.10 last 231.66) - open/close 52166 (time 1746134305.59 total 298.10 last 243.26) - open/close 54740 (time 1746134315.59 total 308.10 last 257.30) - open/close 56680 (time 1746134325.59 total 318.11 last 193.99) - open/close 59314 (time 1746134335.59 total 328.11 last 263.34) - open/close 60000 (time 1746134338.63 total 331.15 last 225.54) - open/close 62161 (time 1746134348.64 total 341.15 last 216.06) - open/close 63954 (time 1746134358.64 total 351.15 last 179.27) total: 64768 open/close in 356.01 seconds: 181.93 ops/second - /unlink 2384 (time 1746134375.05 total 10.00 last 238.36) - /unlink 5142 (time 1746134385.05 total 20.00 last 275.77) - /unlink 8514 (time 1746134395.05 total 30.00 last 337.17) - /unlink 10000 (time 1746134402.08 total 37.04 last 211.23) - /unlink 12327 (time 1746134412.09 total 47.05 last 232.45) - /unlink 15290 (time 1746134422.09 total 57.05 last 296.28) - /unlink 18956 (time 1746134432.10 total 67.05 last 366.56) - /unlink 20000 (time 1746134435.34 total 70.29 last 322.03) - /unlink 23061 (time 1746134445.34 total 80.29 last 306.05) - /unlink 26801 (time 1746134455.34 total 90.30 last 373.94) - /unlink 29659 (time 1746134465.34 total 100.30 last 285.79) - /unlink 32487 (time 1746134475.34 total 110.30 last 282.70) - /unlink 34532 (time 1746134485.35 total 120.30 last 204.47) - /unlink 36927 (time 1746134495.35 total 130.30 last 239.45) - /unlink 39690 (time 1746134505.35 total 140.31 last 276.26) - /unlink 41887 (time 1746134515.35 total 150.31 last 219.70) - /unlink 44076 (time 1746134525.35 total 160.31 last 218.82) - /unlink 46293 (time 1746134535.35 total 170.31 last 221.69) - /unlink 48824 (time 1746134545.36 total 180.31 last 253.00) - /unlink 50000 (time 1746134549.15 total 184.10 last 310.50) - /unlink 52870 (time 1746134559.15 total 194.10 last 286.96) - /unlink 55550 (time 1746134569.15 total 204.11 last 267.94) - /unlink 58515 (time 1746134579.15 total 214.11 last 296.44) - /unlink 60000 (time 1746134583.95 total 218.91 last 309.39) - /unlink 63499 (time 1746134593.95 total 228.91 last 349.86) total: 64768 /unlink in 233.16 seconds: 277.78 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1599 (time 1746134625.03 total 10.01 last 159.76) - open/close 2797 (time 1746134635.04 total 20.02 last 119.71) - open/close 4135 (time 1746134645.04 total 30.02 last 133.77) - open/close 5499 (time 1746134655.04 total 40.02 last 136.38) - open/close 6692 (time 1746134665.05 total 50.03 last 119.22) - open/close 8122 (time 1746134675.05 total 60.03 last 142.99) - open/close 9904 (time 1746134685.06 total 70.03 last 178.12) - open/close 11652 (time 1746134695.07 total 80.04 last 174.63) - open/close 12896 (time 1746134705.08 total 90.05 last 124.26) - open/close 14274 (time 1746134715.09 total 100.06 last 137.66) - open/close 15840 (time 1746134725.09 total 110.06 last 156.57) - open/close 17393 (time 1746134735.09 total 120.07 last 155.26) - open/close 18840 (time 1746134745.09 total 130.07 last 144.69) - open/close 20000 (time 1746134754.79 total 139.77 last 119.61) - open/close 21211 (time 1746134764.80 total 149.77 last 120.99) - open/close 22836 (time 1746134774.80 total 159.78 last 162.46) - open/close 24680 (time 1746134784.81 total 169.78 last 184.32) - open/close 26648 (time 1746134794.81 total 179.78 last 196.74) - open/close 29335 (time 1746134804.81 total 189.79 last 268.68) - open/close 32366 (time 1746134814.82 total 199.79 last 302.98) - open/close 33961 (time 1746134824.82 total 209.79 last 159.48) - open/close 35730 (time 1746134834.82 total 219.79 last 176.90) - open/close 37133 (time 1746134844.82 total 229.80 last 140.22) - open/close 38351 (time 1746134854.82 total 239.80 last 121.79) - open/close 39597 (time 1746134864.83 total 249.81 last 124.48) - open/close 40000 (time 1746134868.92 total 253.90 last 98.50) - open/close 41155 (time 1746134878.93 total 263.90 last 115.46) - open/close 42448 (time 1746134888.94 total 273.91 last 129.19) - open/close 43535 (time 1746134898.94 total 283.92 last 108.61) - open/close 44646 (time 1746134908.95 total 293.92 last 111.04) - open/close 45839 (time 1746134918.95 total 303.93 last 119.28) - open/close 47020 (time 1746134928.95 total 313.93 last 118.06) - open/close 48268 (time 1746134938.96 total 323.93 last 124.79) - open/close 49532 (time 1746134948.96 total 333.94 last 126.33) - open/close 50000 (time 1746134952.47 total 337.44 last 133.38) - open/close 51673 (time 1746134962.48 total 347.45 last 167.20) - open/close 53283 (time 1746134972.48 total 357.45 last 161.00) - open/close 55029 (time 1746134982.48 total 367.46 last 174.51) - open/close 56487 (time 1746134992.49 total 377.46 last 145.74) - open/close 58156 (time 1746135002.49 total 387.47 last 166.81) - open/close 59721 (time 1746135012.50 total 397.47 last 156.43) - open/close 60000 (time 1746135014.50 total 399.47 last 139.45) - open/close 62008 (time 1746135024.50 total 409.47 last 200.78) - open/close 63578 (time 1746135034.50 total 419.48 last 156.90) total: 64768 open/close in 428.65 seconds: 151.10 ops/second - /unlink 3121 (time 1746135054.94 total 10.00 last 312.07) - /unlink 5590 (time 1746135064.94 total 20.00 last 246.90) - /unlink 8180 (time 1746135074.94 total 30.00 last 258.96) - /unlink 10000 (time 1746135080.82 total 35.88 last 309.40) - /unlink 13115 (time 1746135090.83 total 45.89 last 311.45) - /unlink 16622 (time 1746135100.83 total 55.89 last 350.61) - /unlink 19030 (time 1746135110.83 total 65.89 last 240.76) - /unlink 20000 (time 1746135115.27 total 70.33 last 218.27) - /unlink 22234 (time 1746135125.27 total 80.34 last 223.38) - /unlink 25042 (time 1746135135.27 total 90.34 last 280.78) - /unlink 27930 (time 1746135145.28 total 100.34 last 288.78) - /unlink 30000 (time 1746135152.32 total 107.38 last 293.92) - /unlink 33513 (time 1746135162.32 total 117.38 last 351.28) - /unlink 37291 (time 1746135172.32 total 127.38 last 377.78) - /unlink 40000 (time 1746135179.29 total 134.35 last 388.47) - /unlink 43369 (time 1746135189.30 total 144.36 last 336.82) - /unlink 46217 (time 1746135199.30 total 154.36 last 284.73) - /unlink 48646 (time 1746135209.30 total 164.36 last 242.85) - /unlink 50000 (time 1746135214.31 total 169.37 last 270.40) - /unlink 52786 (time 1746135224.31 total 179.37 last 278.52) - /unlink 55020 (time 1746135234.31 total 189.37 last 223.37) - /unlink 57978 (time 1746135244.32 total 199.38 last 295.66) - /unlink 60000 (time 1746135251.97 total 207.03 last 264.34) - /unlink 62816 (time 1746135261.97 total 217.03 last 281.60) total: 64768 /unlink in 223.29 seconds: 290.07 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg438-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg438-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg438-server stop mds service on oleg438-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg438-server unloading modules via unload_modules_local on: 'oleg438-server' oleg438-server: oleg438-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg438-client: oleg438-client: ssh exited with exit code 2 pdsh@oleg438-client: oleg438-server: ssh exited with exit code 2 pdsh@oleg438-client: oleg438-client: ssh exited with exit code 2 pdsh@oleg438-client: oleg438-server: ssh exited with exit code 2 PASS 106 (3674s) Stopping clients: oleg438-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg438-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg438-client: oleg438-server: ssh exited with exit code 2 oleg438-server: oleg438-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg438-server' oleg438-server: oleg438-server.virtnet: executing load_modules_local oleg438-server: Loading modules from /home/green/git/lustre-release/lustre oleg438-server: detected 4 online CPUs by sysfs oleg438-server: MODOPTS_LIBCFS= oleg438-server: Force libcfs to create 2 CPU partitions oleg438-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg438-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg438-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3773 sec ========== 17:35:31 (1746135331) === conf-sanity: start cleanup 17:35:32 (1746135332) === === conf-sanity: finish cleanup 17:35:33 (1746135333) ===