-----============= acceptance-small: conf-sanity ============----- Mon Mar 30 12:27:28 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg620-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg620-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg620-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg620-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg620-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg620-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg620-server oleg620-server: oleg620-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg620-server' oleg620-server: oleg620-server.virtnet: executing load_modules_local oleg620-server: Loading modules from /home/green/git/lustre-release/lustre oleg620-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg620-server: detected 4 online CPUs by sysfs oleg620-server: MODOPTS_LIBCFS= oleg620-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg620-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg620-server: oleg620-server.virtnet: executing set_default_debug -1 all pdsh@oleg620-client: oleg620-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg620-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg620-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg620-server: oleg620-server.virtnet: executing set_default_debug -1 all pdsh@oleg620-client: oleg620-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg620-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg620-server: oleg620-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg620-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg620-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg620-server stop mds service on oleg620-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg620-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 12:29:20 (1774888160) Stopping clients: oleg620-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg620-client.virtnet /mnt/lustre2 (opts:-f) oleg620-server: oleg620-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg620-server' oleg620-server: oleg620-server.virtnet: executing load_modules_local oleg620-server: Loading modules from /home/green/git/lustre-release/lustre oleg620-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg620-server: detected 4 online CPUs by sysfs oleg620-server: MODOPTS_LIBCFS= oleg620-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg620-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg620-server: oleg620-server.virtnet: executing set_default_debug -1 all pdsh@oleg620-client: oleg620-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg620-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg620-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg620-server: oleg620-server.virtnet: executing set_default_debug -1 all pdsh@oleg620-client: oleg620-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg620-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg620-client.virtnet: -o user_xattr,flock 192.168.206.120@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32648 392 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 801 (time 1774888245.94 total 10.01 last 80.04) - open/close 1687 (time 1774888255.94 total 20.01 last 88.57) - open/close 2617 (time 1774888265.95 total 30.02 last 92.92) - open/close 3591 (time 1774888275.95 total 40.02 last 97.40) - open/close 4666 (time 1774888285.95 total 50.02 last 107.50) - open/close 5772 (time 1774888295.96 total 60.03 last 110.49) - open/close 6792 (time 1774888305.97 total 70.04 last 101.91) - open/close 7693 (time 1774888315.97 total 80.04 last 90.07) - open/close 8522 (time 1774888325.98 total 90.05 last 82.83) - open/close 9460 (time 1774888335.98 total 100.05 last 93.79) - open/close 10000 (time 1774888342.89 total 106.96 last 78.14) - open/close 10709 (time 1774888352.89 total 116.96 last 70.90) - open/close 11522 (time 1774888362.90 total 126.96 last 81.28) - open/close 12305 (time 1774888372.90 total 136.97 last 78.27) - open/close 13199 (time 1774888382.90 total 146.97 last 89.38) - open/close 14066 (time 1774888392.91 total 156.98 last 86.65) - open/close 15176 (time 1774888402.91 total 166.98 last 110.93) - open/close 16407 (time 1774888412.92 total 176.99 last 123.03) - open/close 17687 (time 1774888422.92 total 186.99 last 127.96) - open/close 18794 (time 1774888432.92 total 196.99 last 110.69) - open/close 19766 (time 1774888442.92 total 206.99 last 97.20) - open/close 20000 (time 1774888445.32 total 209.39 last 97.54) - open/close 20866 (time 1774888455.33 total 219.39 last 86.58) - open/close 21756 (time 1774888465.34 total 229.40 last 88.90) - open/close 22677 (time 1774888475.34 total 239.41 last 92.04) - open/close 23605 (time 1774888485.35 total 249.42 last 92.74) - open/close 24519 (time 1774888495.37 total 259.43 last 91.27) - open/close 25436 (time 1774888505.37 total 269.44 last 91.62) - open/close 26427 (time 1774888515.38 total 279.44 last 99.09) - open/close 27348 (time 1774888525.38 total 289.44 last 92.10) - open/close 28222 (time 1774888535.38 total 299.44 last 87.38) - open/close 29102 (time 1774888545.39 total 309.45 last 87.93) - open/close 29898 (time 1774888555.39 total 319.46 last 79.55) - open/close 30774 (time 1774888565.39 total 329.46 last 87.58) - open/close 31713 (time 1774888575.41 total 339.48 last 93.76) - open/close 32612 (time 1774888585.41 total 349.48 last 89.85) - open/close 33518 (time 1774888595.42 total 359.48 last 90.57) - open/close 34442 (time 1774888605.42 total 369.49 last 92.40) - open/close 35394 (time 1774888615.42 total 379.49 last 95.16) - open/close 36484 (time 1774888625.43 total 389.49 last 108.94) - open/close 37573 (time 1774888635.43 total 399.50 last 108.85) - open/close 38641 (time 1774888645.43 total 409.50 last 106.79) - open/close 39720 (time 1774888655.44 total 419.50 last 107.86) - open/close 40000 (time 1774888658.33 total 422.39 last 96.86) - open/close 40971 (time 1774888668.33 total 432.40 last 97.06) - open/close 41950 (time 1774888678.33 total 442.40 last 97.87) - open/close 42862 (time 1774888688.34 total 452.40 last 91.17) - open/close 43713 (time 1774888698.34 total 462.41 last 85.10) - open/close 44601 (time 1774888708.35 total 472.42 last 88.71) - open/close 45478 (time 1774888718.36 total 482.43 last 87.58) - open/close 46272 (time 1774888728.38 total 492.45 last 79.27) - open/close 47072 (time 1774888738.39 total 502.46 last 79.92) - open/close 47849 (time 1774888748.39 total 512.46 last 77.67) - open/close 48751 (time 1774888758.39 total 522.46 last 90.20) - open/close 49676 (time 1774888768.40 total 532.47 last 92.42) - open/close 50000 (time 1774888772.33 total 536.40 last 82.48) - open/close 50861 (time 1774888782.33 total 546.40 last 86.08) - open/close 51768 (time 1774888792.34 total 556.41 last 90.64) - open/close 52640 (time 1774888802.34 total 566.41 last 87.16) - open/close 53512 (time 1774888812.35 total 576.42 last 87.13) - open/close 54364 (time 1774888822.36 total 586.43 last 85.12) - open/close 55194 (time 1774888832.37 total 596.44 last 82.90) - open/close 56057 (time 1774888842.38 total 606.45 last 86.21) - open/close 56915 (time 1774888852.40 total 616.47 last 85.66) - open/close 57879 (time 1774888862.40 total 626.47 last 96.38) - open/close 58921 (time 1774888872.40 total 636.47 last 104.19) - open/close 59879 (time 1774888882.41 total 646.48 last 95.73) - open/close 60734 (time 1774888892.43 total 656.49 last 85.37) - open/close 61475 (time 1774888902.44 total 666.51 last 73.99) - open/close 62265 (time 1774888912.45 total 676.51 last 78.94) - open/close 63060 (time 1774888922.45 total 686.52 last 79.44) - open/close 63946 (time 1774888932.46 total 696.52 last 88.58) - open/close 64757 (time 1774888942.46 total 706.52 last 81.09) total: 64768 open/close in 706.66 seconds: 91.65 ops/second - /unlink 762 (time 1774888955.88 total 10.00 last 76.18) - /unlink 1567 (time 1774888965.89 total 20.01 last 80.45) - /unlink 2459 (time 1774888975.89 total 30.02 last 89.12) - /unlink 3240 (time 1774888985.90 total 40.02 last 78.06) - /unlink 4114 (time 1774888995.90 total 50.03 last 87.38) - /unlink 5001 (time 1774889005.90 total 60.03 last 88.69) - /unlink 5836 (time 1774889015.90 total 70.03 last 83.49) - /unlink 6688 (time 1774889025.91 total 80.04 last 85.11) - /unlink 7518 (time 1774889035.92 total 90.05 last 82.93) - /unlink 8425 (time 1774889045.93 total 100.06 last 90.60) - /unlink 9249 (time 1774889055.93 total 110.06 last 82.40) - /unlink 10000 (time 1774889064.85 total 118.97 last 84.25) - /unlink 10794 (time 1774889074.86 total 128.98 last 79.34) - /unlink 11562 (time 1774889084.86 total 138.99 last 76.74) - /unlink 12318 (time 1774889094.87 total 148.99 last 75.54) - /unlink 13114 (time 1774889104.87 total 159.00 last 79.58) - /unlink 14000 (time 1774889114.88 total 169.00 last 88.54) - /unlink 14906 (time 1774889124.89 total 179.01 last 90.52) - /unlink 15876 (time 1774889134.89 total 189.02 last 96.94) - /unlink 16794 (time 1774889144.90 total 199.02 last 91.76) - /unlink 17678 (time 1774889154.91 total 209.03 last 88.32) - /unlink 18589 (time 1774889164.92 total 219.04 last 91.01) - /unlink 19511 (time 1774889174.92 total 229.05 last 92.16) - /unlink 20000 (time 1774889180.69 total 234.81 last 84.81) - /unlink 20807 (time 1774889190.70 total 244.82 last 80.64) - /unlink 21544 (time 1774889200.70 total 254.82 last 73.70) - /unlink 22326 (time 1774889210.70 total 264.83 last 78.15) - /unlink 23145 (time 1774889220.71 total 274.84 last 81.83) - /unlink 23933 (time 1774889230.72 total 284.84 last 78.78) - /unlink 24855 (time 1774889240.72 total 294.84 last 92.18) - /unlink 25701 (time 1774889250.73 total 304.85 last 84.53) - /unlink 26583 (time 1774889260.73 total 314.86 last 88.14) - /unlink 27383 (time 1774889270.74 total 324.87 last 79.91) - /unlink 28175 (time 1774889280.75 total 334.87 last 79.15) - /unlink 28968 (time 1774889290.76 total 344.88 last 79.22) - /unlink 29733 (time 1774889300.77 total 354.89 last 76.46) - /unlink 30000 (time 1774889303.67 total 357.79 last 92.06) - /unlink 30861 (time 1774889313.67 total 367.80 last 86.05) - /unlink 31682 (time 1774889323.68 total 377.80 last 82.07) - /unlink 32560 (time 1774889333.68 total 387.81 last 87.75) - /unlink 33387 (time 1774889343.71 total 397.83 last 82.49) - /unlink 34321 (time 1774889353.71 total 407.84 last 93.34) - /unlink 35226 (time 1774889363.72 total 417.84 last 90.44) - /unlink 36326 (time 1774889373.72 total 427.85 last 109.98) - /unlink 37256 (time 1774889383.74 total 437.86 last 92.88) - /unlink 38157 (time 1774889393.74 total 447.86 last 90.08) - /unlink 39077 (time 1774889403.74 total 457.86 last 91.99) - /unlink 40000 (time 1774889413.36 total 467.49 last 95.91) - /unlink 40907 (time 1774889423.36 total 477.49 last 90.68) - /unlink 41883 (time 1774889433.37 total 487.49 last 97.58) - /unlink 42913 (time 1774889443.37 total 497.49 last 102.99) - /unlink 43891 (time 1774889453.37 total 507.50 last 97.74) - /unlink 44956 (time 1774889463.38 total 517.50 last 106.48) - /unlink 45920 (time 1774889473.38 total 527.50 last 96.39) - /unlink 47038 (time 1774889483.38 total 537.50 last 111.80) - /unlink 48179 (time 1774889493.41 total 547.53 last 113.77) - /unlink 49101 (time 1774889503.41 total 557.53 last 92.17) - /unlink 49884 (time 1774889513.42 total 567.55 last 78.20) - /unlink 50816 (time 1774889523.42 total 577.55 last 93.18) - /unlink 51732 (time 1774889533.43 total 587.55 last 91.54) - /unlink 52678 (time 1774889543.44 total 597.56 last 94.54) - /unlink 53618 (time 1774889553.44 total 607.56 last 93.99) - /unlink 54566 (time 1774889563.45 total 617.57 last 94.71) - /unlink 55435 (time 1774889573.45 total 627.57 last 86.87) - /unlink 56308 (time 1774889583.45 total 637.58 last 87.29) - /unlink 57262 (time 1774889593.46 total 647.58 last 95.36) - /unlink 58213 (time 1774889603.46 total 657.58 last 95.09) - /unlink 59122 (time 1774889613.46 total 667.59 last 90.85) - /unlink 60000 (time 1774889622.04 total 676.17 last 102.34) - /unlink 60967 (time 1774889632.05 total 686.17 last 96.62) - /unlink 61873 (time 1774889642.05 total 696.17 last 90.59) - /unlink 62778 (time 1774889652.07 total 706.19 last 90.33) - /unlink 63787 (time 1774889662.08 total 716.20 last 100.83) - /unlink 64658 (time 1774889672.08 total 726.21 last 87.04) total: 64768 /unlink in 727.37 seconds: 89.04 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1029 (time 1774889706.52 total 10.00 last 102.85) - open/close 2092 (time 1774889716.52 total 20.01 last 106.21) - open/close 3171 (time 1774889726.53 total 30.02 last 107.83) - open/close 4129 (time 1774889736.53 total 40.02 last 95.79) - open/close 5065 (time 1774889746.54 total 50.03 last 93.55) - open/close 6048 (time 1774889756.54 total 60.03 last 98.30) - open/close 6922 (time 1774889766.54 total 70.03 last 87.37) - open/close 7899 (time 1774889776.55 total 80.04 last 97.62) - open/close 8926 (time 1774889786.56 total 90.05 last 102.56) - open/close 9877 (time 1774889796.56 total 100.05 last 95.08) - open/close 10843 (time 1774889806.57 total 110.06 last 96.55) - open/close 11736 (time 1774889816.58 total 120.07 last 89.22) - open/close 12652 (time 1774889826.58 total 130.07 last 91.60) - open/close 13717 (time 1774889836.58 total 140.07 last 106.50) - open/close 14819 (time 1774889846.58 total 150.07 last 110.14) - open/close 15962 (time 1774889856.58 total 160.07 last 114.28) - open/close 16986 (time 1774889866.59 total 170.08 last 102.38) - open/close 18076 (time 1774889876.59 total 180.08 last 108.95) - open/close 19209 (time 1774889886.59 total 190.08 last 113.28) - open/close 20000 (time 1774889892.84 total 196.33 last 126.71) - open/close 21071 (time 1774889902.85 total 206.34 last 106.99) - open/close 22079 (time 1774889912.85 total 216.34 last 100.71) - open/close 23071 (time 1774889922.86 total 226.35 last 99.19) - open/close 24095 (time 1774889932.86 total 236.35 last 102.35) - open/close 25136 (time 1774889942.88 total 246.37 last 103.90) - open/close 26121 (time 1774889952.89 total 256.38 last 98.43) - open/close 27094 (time 1774889962.89 total 266.38 last 97.26) - open/close 28107 (time 1774889972.90 total 276.39 last 101.25) - open/close 29030 (time 1774889982.91 total 286.40 last 92.17) - open/close 30000 (time 1774889992.60 total 296.09 last 100.05) - open/close 30958 (time 1774890002.61 total 306.10 last 95.72) - open/close 31843 (time 1774890012.62 total 316.11 last 88.41) - open/close 32781 (time 1774890022.63 total 326.12 last 93.69) - open/close 33718 (time 1774890032.65 total 336.14 last 93.55) - open/close 34638 (time 1774890042.66 total 346.15 last 91.92) - open/close 35559 (time 1774890052.66 total 356.15 last 92.05) - open/close 36594 (time 1774890062.67 total 366.16 last 103.43) - open/close 37682 (time 1774890072.67 total 376.16 last 108.78) - open/close 38706 (time 1774890082.67 total 386.16 last 102.38) - open/close 39700 (time 1774890092.68 total 396.16 last 99.39) - open/close 40000 (time 1774890095.65 total 399.14 last 100.84) - open/close 40977 (time 1774890105.65 total 409.14 last 97.70) - open/close 41916 (time 1774890115.66 total 419.15 last 93.83) - open/close 42813 (time 1774890125.66 total 429.15 last 89.66) - open/close 43738 (time 1774890135.67 total 439.16 last 92.39) - open/close 44837 (time 1774890145.68 total 449.17 last 109.88) - open/close 45877 (time 1774890155.68 total 459.17 last 103.93) - open/close 46805 (time 1774890165.71 total 469.20 last 92.60) - open/close 47780 (time 1774890175.71 total 479.20 last 97.49) - open/close 49078 (time 1774890185.71 total 489.20 last 129.69) - open/close 50000 (time 1774890192.97 total 496.46 last 127.13) - open/close 51163 (time 1774890202.97 total 506.46 last 116.29) - open/close 52212 (time 1774890212.97 total 516.46 last 104.89) - open/close 53296 (time 1774890222.97 total 526.46 last 108.38) - open/close 54456 (time 1774890232.98 total 536.47 last 115.90) - open/close 55582 (time 1774890242.99 total 546.48 last 112.52) - open/close 56786 (time 1774890252.99 total 556.48 last 120.29) - open/close 57833 (time 1774890263.01 total 566.50 last 104.56) - open/close 59007 (time 1774890273.01 total 576.50 last 117.33) - open/close 60000 (time 1774890282.47 total 585.96 last 105.00) - open/close 61130 (time 1774890292.47 total 595.96 last 112.97) - open/close 62236 (time 1774890302.48 total 605.97 last 110.58) - open/close 63597 (time 1774890312.48 total 615.97 last 136.00) - open/close 64652 (time 1774890322.49 total 625.98 last 105.45) total: 64768 open/close in 626.93 seconds: 103.31 ops/second - /unlink 1315 (time 1774890335.58 total 10.00 last 131.46) - /unlink 2449 (time 1774890345.58 total 20.01 last 113.37) - /unlink 3475 (time 1774890355.58 total 30.01 last 102.56) - /unlink 4594 (time 1774890365.59 total 40.02 last 111.83) - /unlink 5664 (time 1774890375.60 total 50.03 last 106.86) - /unlink 6762 (time 1774890385.60 total 60.03 last 109.77) - /unlink 7896 (time 1774890395.61 total 70.03 last 113.36) - /unlink 9184 (time 1774890405.61 total 80.04 last 128.78) - /unlink 10000 (time 1774890412.51 total 86.94 last 118.20) - /unlink 11081 (time 1774890422.52 total 96.94 last 108.04) - /unlink 12209 (time 1774890432.52 total 106.95 last 112.77) - /unlink 13449 (time 1774890442.52 total 116.95 last 123.96) - /unlink 14630 (time 1774890452.53 total 126.95 last 118.06) - /unlink 15665 (time 1774890462.53 total 136.96 last 103.42) - /unlink 16754 (time 1774890472.54 total 146.97 last 108.86) - /unlink 17818 (time 1774890482.54 total 156.97 last 106.35) - /unlink 18925 (time 1774890492.55 total 166.97 last 110.66) - /unlink 20000 (time 1774890501.59 total 176.02 last 118.81) - /unlink 21527 (time 1774890511.60 total 186.02 last 152.67) - /unlink 22912 (time 1774890521.60 total 196.03 last 138.48) - /unlink 25329 (time 1774890531.60 total 206.03 last 241.63) - /unlink 27758 (time 1774890541.60 total 216.03 last 242.83) - /unlink 30000 (time 1774890549.52 total 223.94 last 283.41) - /unlink 33061 (time 1774890559.52 total 233.95 last 306.02) - /unlink 35933 (time 1774890569.52 total 243.95 last 287.17) - /unlink 38493 (time 1774890579.53 total 253.96 last 255.74) - /unlink 40000 (time 1774890585.88 total 260.31 last 237.13) - /unlink 42310 (time 1774890595.89 total 270.32 last 230.92) - /unlink 44596 (time 1774890605.89 total 280.32 last 228.55) - /unlink 46834 (time 1774890615.89 total 290.32 last 223.79) - /unlink 48983 (time 1774890625.89 total 300.32 last 214.84) - /unlink 50000 (time 1774890629.87 total 304.30 last 255.85) - /unlink 52709 (time 1774890639.87 total 314.30 last 270.83) - /unlink 55821 (time 1774890649.87 total 324.30 last 311.17) - /unlink 58886 (time 1774890659.87 total 334.30 last 306.44) - /unlink 60000 (time 1774890662.84 total 337.26 last 376.19) - /unlink 62934 (time 1774890672.84 total 347.27 last 293.31) total: 64768 /unlink in 353.69 seconds: 183.12 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1523 (time 1774890713.81 total 10.00 last 152.24) - open/close 3216 (time 1774890723.81 total 20.00 last 169.29) - open/close 5202 (time 1774890733.82 total 30.01 last 198.53) - open/close 7066 (time 1774890743.82 total 40.01 last 186.36) - open/close 9192 (time 1774890753.82 total 50.01 last 212.57) - open/close 10000 (time 1774890756.67 total 52.86 last 283.33) - open/close 12318 (time 1774890766.67 total 62.86 last 231.79) - open/close 14600 (time 1774890776.67 total 72.87 last 228.17) - open/close 16253 (time 1774890786.68 total 82.87 last 165.28) - open/close 18784 (time 1774890796.68 total 92.87 last 253.10) - open/close 20000 (time 1774890801.44 total 97.63 last 255.34) - open/close 21482 (time 1774890811.44 total 107.63 last 148.15) - open/close 23153 (time 1774890821.44 total 117.63 last 167.07) - open/close 25055 (time 1774890831.45 total 127.64 last 190.12) - open/close 27050 (time 1774890841.45 total 137.64 last 199.47) - open/close 29699 (time 1774890851.45 total 147.64 last 264.85) - open/close 32331 (time 1774890861.46 total 157.65 last 263.09) - open/close 34413 (time 1774890871.46 total 167.65 last 208.11) - open/close 36311 (time 1774890881.46 total 177.65 last 189.79) - open/close 38529 (time 1774890891.46 total 187.65 last 221.77) - open/close 40000 (time 1774890898.07 total 194.26 last 222.69) - open/close 42442 (time 1774890908.07 total 204.26 last 244.18) - open/close 44610 (time 1774890918.07 total 214.26 last 216.68) - open/close 46766 (time 1774890928.08 total 224.27 last 215.57) - open/close 49115 (time 1774890938.08 total 234.27 last 234.87) - open/close 50000 (time 1774890941.55 total 237.74 last 254.61) - open/close 52219 (time 1774890951.55 total 247.74 last 221.89) - open/close 54116 (time 1774890961.56 total 257.75 last 189.65) - open/close 56076 (time 1774890971.56 total 267.75 last 195.95) - open/close 58109 (time 1774890981.56 total 277.75 last 203.28) - open/close 60000 (time 1774890990.95 total 287.14 last 201.34) - open/close 62294 (time 1774891000.96 total 297.15 last 229.30) - open/close 64616 (time 1774891010.96 total 307.15 last 232.04) total: 64768 open/close in 308.02 seconds: 210.27 ops/second - /unlink 3920 (time 1774891023.42 total 10.00 last 391.96) - /unlink 7200 (time 1774891033.42 total 20.00 last 327.93) - /unlink 10000 (time 1774891041.80 total 28.38 last 334.06) - /unlink 14174 (time 1774891051.81 total 38.39 last 417.18) - /unlink 18038 (time 1774891061.81 total 48.39 last 386.38) - /unlink 20000 (time 1774891065.78 total 52.37 last 493.31) - /unlink 24705 (time 1774891075.78 total 62.37 last 470.49) - /unlink 29664 (time 1774891085.79 total 72.37 last 495.81) - /unlink 34168 (time 1774891095.79 total 82.37 last 450.35) - /unlink 39636 (time 1774891105.79 total 92.37 last 546.77) - /unlink 44456 (time 1774891115.79 total 102.37 last 481.93) - /unlink 49238 (time 1774891125.79 total 112.37 last 478.15) - /unlink 54160 (time 1774891135.79 total 122.38 last 492.15) - /unlink 58676 (time 1774891145.79 total 132.38 last 451.55) - /unlink 60000 (time 1774891148.97 total 135.56 last 416.14) - /unlink 64557 (time 1774891158.98 total 145.56 last 455.60) total: 64768 /unlink in 146.22 seconds: 442.95 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2788 (time 1774891186.58 total 10.00 last 278.73) - open/close 5451 (time 1774891196.58 total 20.00 last 266.27) - open/close 8159 (time 1774891206.58 total 30.01 last 270.72) - open/close 10000 (time 1774891211.79 total 35.21 last 353.79) - open/close 13325 (time 1774891221.79 total 45.21 last 332.47) - open/close 16310 (time 1774891231.79 total 55.21 last 298.49) - open/close 19671 (time 1774891241.79 total 65.21 last 336.03) - open/close 22829 (time 1774891251.79 total 75.22 last 315.75) - open/close 26769 (time 1774891261.79 total 85.22 last 393.99) - open/close 30000 (time 1774891269.10 total 92.52 last 442.37) - open/close 33741 (time 1774891279.10 total 102.52 last 374.09) - open/close 37366 (time 1774891289.10 total 112.52 last 362.44) - open/close 40000 (time 1774891296.39 total 119.81 last 361.16) - open/close 43889 (time 1774891306.39 total 129.82 last 388.84) - open/close 48012 (time 1774891316.39 total 139.82 last 412.29) - open/close 50000 (time 1774891320.29 total 143.71 last 510.16) - open/close 54217 (time 1774891330.29 total 153.71 last 421.62) - open/close 58344 (time 1774891340.29 total 163.72 last 412.62) - open/close 60000 (time 1774891344.91 total 168.33 last 358.91) - open/close 63573 (time 1774891354.91 total 178.33 last 357.29) total: 64768 open/close in 181.33 seconds: 357.19 ops/second - /unlink 7134 (time 1774891369.36 total 10.00 last 713.34) - /unlink 10000 (time 1774891373.37 total 14.01 last 714.70) - /unlink 14828 (time 1774891383.38 total 24.01 last 482.67) - /unlink 20000 (time 1774891392.10 total 32.74 last 592.93) - /unlink 26385 (time 1774891402.10 total 42.74 last 638.43) - /unlink 30000 (time 1774891407.78 total 48.42 last 636.20) - /unlink 35456 (time 1774891417.78 total 58.42 last 545.59) - /unlink 40000 (time 1774891424.50 total 65.14 last 676.14) - /unlink 46607 (time 1774891434.50 total 75.14 last 660.62) - /unlink 50000 (time 1774891440.31 total 80.95 last 584.64) - /unlink 56424 (time 1774891450.31 total 90.95 last 642.39) - /unlink 60000 (time 1774891454.85 total 95.49 last 786.98) total: 64768 /unlink in 103.20 seconds: 627.60 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 4458 (time 1774891495.93 total 10.00 last 445.79) - open/close 8371 (time 1774891505.93 total 20.00 last 391.24) - open/close 10000 (time 1774891510.78 total 24.85 last 336.04) - open/close 14200 (time 1774891520.78 total 34.85 last 420.00) - open/close 18326 (time 1774891530.78 total 44.85 last 412.52) - open/close 20000 (time 1774891534.50 total 48.57 last 450.38) - open/close 23745 (time 1774891544.50 total 58.57 last 374.47) - open/close 27044 (time 1774891554.50 total 68.57 last 329.83) - open/close 30000 (time 1774891562.99 total 77.06 last 348.12) - open/close 33284 (time 1774891572.99 total 87.06 last 328.32) - open/close 36177 (time 1774891582.99 total 97.07 last 289.25) - open/close 39660 (time 1774891593.00 total 107.07 last 348.24) - open/close 42650 (time 1774891603.00 total 117.07 last 298.95) - open/close 46353 (time 1774891613.00 total 127.07 last 370.24) - open/close 50000 (time 1774891622.94 total 137.02 last 366.70) - open/close 53445 (time 1774891632.94 total 147.02 last 344.50) - open/close 57206 (time 1774891642.95 total 157.02 last 376.04) - open/close 60000 (time 1774891649.77 total 163.85 last 409.21) - open/close 64056 (time 1774891659.77 total 173.85 last 405.57) total: 64768 open/close in 175.36 seconds: 369.34 ops/second - /unlink 7482 (time 1774891672.67 total 10.00 last 748.11) - /unlink 10000 (time 1774891676.02 total 13.35 last 751.15) - /unlink 17287 (time 1774891686.03 total 23.35 last 728.59) - /unlink 20000 (time 1774891689.34 total 26.67 last 819.47) - /unlink 26681 (time 1774891699.34 total 36.67 last 668.09) - /unlink 30000 (time 1774891704.32 total 41.65 last 666.05) - /unlink 37113 (time 1774891714.32 total 51.65 last 711.22) - /unlink 40000 (time 1774891718.86 total 56.19 last 636.25) - /unlink 47023 (time 1774891728.86 total 66.19 last 702.25) - /unlink 50000 (time 1774891733.48 total 70.81 last 644.51) - /unlink 56228 (time 1774891743.48 total 80.81 last 622.73) - /unlink 60000 (time 1774891749.54 total 86.87 last 622.40) total: 64768 /unlink in 95.18 seconds: 680.51 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg620-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg620-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg620-server stop mds service on oleg620-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg620-server unloading modules via unload_modules_local on: 'oleg620-server' oleg620-server: oleg620-server.virtnet: executing unload_modules_local oleg620-server: modules unloaded. pdsh@oleg620-client: oleg620-client: ssh exited with exit code 2 pdsh@oleg620-client: oleg620-server: ssh exited with exit code 2 pdsh@oleg620-client: oleg620-client: ssh exited with exit code 2 pdsh@oleg620-client: oleg620-server: ssh exited with exit code 2 PASS 106 (3625s) Stopping clients: oleg620-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg620-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg620-client: oleg620-server: ssh exited with exit code 2 oleg620-server: oleg620-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg620-server' oleg620-server: oleg620-server.virtnet: executing load_modules_local oleg620-server: Loading modules from /home/green/git/lustre-release/lustre oleg620-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg620-server: detected 4 online CPUs by sysfs oleg620-server: MODOPTS_LIBCFS= oleg620-server: Force libcfs to create 2 CPU partitions oleg620-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg620-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg620-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3760 sec ========== 13:30:10 (1774891810) === conf-sanity: start cleanup 13:30:11 (1774891811) === === conf-sanity: finish cleanup 13:30:12 (1774891812) ===