-----============= acceptance-small: conf-sanity ============----- Thu Apr 23 00:42:49 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg415-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball Stopping clients: oleg415-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg415-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg415-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg415-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg415-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg415-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg415-server oleg415-server: oleg415-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: detected 4 online CPUs by sysfs oleg415-server: MODOPTS_LIBCFS= oleg415-server: Force libcfs to create 2 CPU partitions oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg415-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg415-server: oleg415-server.virtnet: executing set_default_debug -1 all pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg415-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg415-server: oleg415-server.virtnet: executing set_default_debug -1 all pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg415-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg415-server: oleg415-server.virtnet: executing set_default_debug -1 all pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg415-server: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg415-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg415-server: oleg415-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg415-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg415-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg415-server stop mds service on oleg415-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg415-server stop mds service on oleg415-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg415-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 00:45:07 (1776919507) start mds service on oleg415-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg415-server: oleg415-server.virtnet: executing set_default_debug -1 all pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg415-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg415-server: oleg415-server.virtnet: executing set_default_debug -1 all pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg415-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg415-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg415-server: oleg415-server.virtnet: executing set_default_debug -1 all pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 Started lustre-OST0000 oleg415-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg415-client.virtnet: -o user_xattr,flock 192.168.204.115@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 50000 432 49568 1% /mnt/lustre[OST:0] filesystem_summary: 50097 529 49568 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close/unlink 712 (time 1776919561.66 total 10.01 last 71.12) - open/close/unlink 1361 (time 1776919571.67 total 20.02 last 64.85) - open/close/unlink 2036 (time 1776919581.68 total 30.03 last 67.41) - open/close/unlink 2728 (time 1776919591.69 total 40.04 last 69.14) - open/close/unlink 3482 (time 1776919601.70 total 50.05 last 75.31) - open/close/unlink 4221 (time 1776919611.71 total 60.06 last 73.84) - open/close/unlink 4946 (time 1776919621.72 total 70.08 last 72.41) - open/close/unlink 5622 (time 1776919631.73 total 80.08 last 67.58) - open/close/unlink 6212 (time 1776919641.74 total 90.09 last 58.93) - open/close/unlink 6731 (time 1776919651.75 total 100.10 last 51.83) - open/close/unlink 7372 (time 1776919661.76 total 110.11 last 64.04) - open/close/unlink 8047 (time 1776919671.77 total 120.12 last 67.44) - open/close/unlink 8755 (time 1776919681.78 total 130.13 last 70.75) - open/close/unlink 9445 (time 1776919691.80 total 140.15 last 68.86) - open/close/unlink 10000 (time 1776919701.23 total 149.58 last 58.84) - open/close/unlink 10911 (time 1776919711.24 total 159.59 last 91.03) - open/close/unlink 11859 (time 1776919721.24 total 169.60 last 94.75) - open/close/unlink 12628 (time 1776919731.26 total 179.61 last 76.77) - open/close/unlink 13673 (time 1776919741.27 total 189.62 last 104.43) - open/close/unlink 14579 (time 1776919751.28 total 199.63 last 90.53) - open/close/unlink 15481 (time 1776919761.29 total 209.64 last 90.10) - open/close/unlink 16417 (time 1776919771.34 total 219.69 last 93.15) - open/close/unlink 17363 (time 1776919781.35 total 229.70 last 94.50) - open/close/unlink 18307 (time 1776919791.35 total 239.70 last 94.40) - open/close/unlink 19508 (time 1776919801.35 total 249.70 last 120.02) - open/close/unlink 20000 (time 1776919805.87 total 254.22 last 108.85) - open/close/unlink 21034 (time 1776919815.88 total 264.23 last 103.36) - open/close/unlink 21994 (time 1776919825.88 total 274.23 last 95.98) - open/close/unlink 23001 (time 1776919835.89 total 284.24 last 100.63) - open/close/unlink 23871 (time 1776919845.89 total 294.24 last 87.00) - open/close/unlink 24875 (time 1776919855.89 total 304.24 last 100.40) - open/close/unlink 25883 (time 1776919865.89 total 314.24 last 100.80) - open/close/unlink 26858 (time 1776919875.89 total 324.24 last 97.46) - open/close/unlink 27810 (time 1776919885.89 total 334.24 last 95.18) - open/close/unlink 28815 (time 1776919895.90 total 344.25 last 100.40) - open/close/unlink 29620 (time 1776919905.91 total 354.26 last 80.48) - open/close/unlink 30000 (time 1776919909.47 total 357.82 last 106.71) - open/close/unlink 30853 (time 1776919919.48 total 367.83 last 85.21) - open/close/unlink 31686 (time 1776919929.48 total 377.83 last 83.26) - open/close/unlink 32483 (time 1776919939.48 total 387.83 last 79.68) - open/close/unlink 33264 (time 1776919949.49 total 397.84 last 78.08) - open/close/unlink 34167 (time 1776919959.49 total 407.84 last 90.28) - open/close/unlink 35096 (time 1776919969.49 total 417.84 last 92.87) - open/close/unlink 35971 (time 1776919979.49 total 427.84 last 87.49) - open/close/unlink 36755 (time 1776919989.49 total 437.84 last 78.40) - open/close/unlink 37594 (time 1776919999.49 total 447.84 last 83.90) - open/close/unlink 38365 (time 1776920009.50 total 457.85 last 77.02) - open/close/unlink 39180 (time 1776920019.51 total 467.86 last 81.42) - open/close/unlink 40000 (time 1776920029.22 total 477.57 last 84.52) - open/close/unlink 40807 (time 1776920039.22 total 487.57 last 80.68) - open/close/unlink 41733 (time 1776920049.22 total 497.57 last 92.59) - open/close/unlink 42721 (time 1776920059.22 total 507.57 last 98.77) - open/close/unlink 43683 (time 1776920069.23 total 517.58 last 96.16) - open/close/unlink 44490 (time 1776920079.23 total 527.58 last 80.65) - open/close/unlink 45265 (time 1776920089.25 total 537.60 last 77.40) - open/close/unlink 46046 (time 1776920099.25 total 547.60 last 78.09) - open/close/unlink 46926 (time 1776920109.25 total 557.60 last 87.95) - open - open/close/unlink 708 (time 1776919561.89 total 10.01 last 70.75) - open/close/unlink 1354 (time 1776919571.89 total 20.01 last 64.58) - open/close/unlink 2039 (time 1776919581.90 total 30.02 last 68.47) - open/close/unlink 2732 (time 1776919591.92 total 40.04 last 69.14) - open/close/unlink 3490 (time 1776919601.94 total 50.05 last 75.69) - open/close/unlink 4226 (time 1776919611.95 total 60.07 last 73.49) - open/close/unlink 4954 (time 1776919621.97 total 70.09 last 72.63) - open/close/unlink 5636 (time 1776919632.00 total 80.12 last 68.01) - open/close/unlink 6227 (time 1776919642.00 total 90.12 last 59.09) - open/close/unlink 6748 (time 1776919652.01 total 100.13 last 52.06) - open/close/unlink 7394 (time 1776919662.02 total 110.14 last 64.52) - open/close/unlink 8066 (time 1776919672.03 total 120.15 last 67.17) - open/close/unlink 8768 (time 1776919682.03 total 130.15 last 70.16) - open/close/unlink 9459 (time 1776919692.05 total 140.17 last 68.99) - open/close/unlink 10000 (time 1776919701.35 total 149.47 last 58.17) - open/close/unlink 10911 (time 1776919711.35 total 159.47 last 91.08) - open/close/unlink 11849 (time 1776919721.36 total 169.48 last 93.73) - open/close/unlink 12623 (time 1776919731.36 total 179.48 last 77.39) - open/close/unlink 13675 (time 1776919741.36 total 189.48 last 105.19) - open/close/unlink 14567 (time 1776919751.37 total 199.48 last 89.15) - open/close/unlink 15469 (time 1776919761.37 total 209.49 last 90.18) - open/close/unlink 16401 (time 1776919771.38 total 219.50 last 93.12) - open/close/unlink 17349 (time 1776919781.39 total 229.50 last 94.75) - open/close/unlink 18289 (time 1776919791.39 total 239.50 last 94.00) - open/close/unlink 19492 (time 1776919801.40 total 249.52 last 120.11) - open/close/unlink 20000 (time 1776919806.09 total 254.21 last 108.33) - open/close/unlink 21025 (time 1776919816.09 total 264.21 last 102.49) - open/close/unlink 22006 (time 1776919826.10 total 274.22 last 97.99) - open/close/unlink 23002 (time 1776919836.11 total 284.23 last 99.51) - open/close/unlink 23862 (time 1776919846.13 total 294.24 last 85.88) - open/close/unlink 24876 (time 1776919856.13 total 304.25 last 101.36) - open/close/unlink 25866 (time 1776919866.14 total 314.25 last 98.93) - open/close/unlink 26857 (time 1776919876.14 total 324.25 last 99.10) - open/close/unlink 27807 (time 1776919886.14 total 334.26 last 94.97) - open/close/unlink 28805 (time 1776919896.15 total 344.27 last 99.69) - open/close/unlink 29604 (time 1776919906.16 total 354.28 last 79.79) - open/close/unlink 30000 (time 1776919909.98 total 358.10 last 103.70) - open/close/unlink 30840 (time 1776919919.99 total 368.11 last 83.95) - open/close/unlink 31665 (time 1776919930.00 total 378.11 last 82.43) - open/close/unlink 32452 (time 1776919940.00 total 388.12 last 78.69) - open/close/unlink 33240 (time 1776919950.00 total 398.12 last 78.78) - open/close/unlink 34163 (time 1776919960.01 total 408.12 last 92.26) - open/close/unlink 35088 (time 1776919970.01 total 418.13 last 92.42) - open/close/unlink 35956 (time 1776919980.03 total 428.14 last 86.71) - open/close/unlink 36751 (time 1776919990.03 total 438.15 last 79.43) - open/close/unlink 37583 (time 1776920000.04 total 448.15 last 83.19) - open/close/unlink 38341 (time 1776920010.04 total 458.15 last 75.79) - open/close/unlink 39171 (time 1776920020.05 total 468.17 last 82.88) - open/close/unlink 40000 (time 1776920029.83 total 477.95 last 84.74) - open/close/unlink 40812 (time 1776920039.83 total 487.95 last 81.20) - open/close/unlink 41737 (time 1776920049.86 total 497.97 last 92.30) - open/close/unlink 42725 (time 1776920059.86 total 507.97 last 98.78) - open/close/unlink 43680 (time 1776920069.87 total 517.98 last 95.43) - open/close/unlink 44489 (time 1776920079.87 total 527.99 last 80.86) - open/close/unlink 45257 (time 1776920089.87 total 537.99 last 76.80) - open/close/unlink 46060 (time 1776920099.87 total 547.99 last 80.28) - open/close/unlink 46947 (time 1776920109.89 total 558.00 last 88.59) - open/cl - open/close/unlink 715 (time 1776919561.88 total 10.01 last 71.43) - open/close/unlink 1366 (time 1776919571.89 total 20.02 last 65.03) - open/close/unlink 2045 (time 1776919581.89 total 30.02 last 67.88) - open/close/unlink 2727 (time 1776919591.91 total 40.04 last 68.05) - open/close/unlink 3477 (time 1776919601.94 total 50.07 last 74.82) - open/close/unlink 4205 (time 1776919611.94 total 60.07 last 72.76) - open/close/unlink 4928 (time 1776919621.94 total 70.07 last 72.30) - open/close/unlink 5615 (time 1776919631.96 total 80.10 last 68.55) - open/close/unlink 6205 (time 1776919641.99 total 90.12 last 58.84) - open/close/unlink 6727 (time 1776919652.00 total 100.13 last 52.16) - open/close/unlink 7371 (time 1776919662.03 total 110.16 last 64.19) - open/close/unlink 8046 (time 1776919672.04 total 120.17 last 67.47) - open/close/unlink 8750 (time 1776919682.04 total 130.17 last 70.35) - open/close/unlink 9439 (time 1776919692.06 total 140.19 last 68.77) - open/close/unlink 10000 (time 1776919701.74 total 149.87 last 57.97) - open/close/unlink 10936 (time 1776919711.74 total 159.87 last 93.57) - open/close/unlink 11858 (time 1776919721.74 total 169.88 last 92.17) - open/close/unlink 12639 (time 1776919731.75 total 179.88 last 78.05) - open/close/unlink 13710 (time 1776919741.76 total 189.89 last 107.05) - open/close/unlink 14578 (time 1776919751.76 total 199.89 last 86.80) - open/close/unlink 15491 (time 1776919761.76 total 209.90 last 91.23) - open/close/unlink 16423 (time 1776919771.77 total 219.90 last 93.16) - open/close/unlink 17357 (time 1776919781.78 total 229.91 last 93.32) - open/close/unlink 18307 (time 1776919791.78 total 239.91 last 94.98) - open/close/unlink 19503 (time 1776919801.78 total 249.92 last 119.57) - open/close/unlink 20000 (time 1776919806.44 total 254.57 last 106.81) - open/close/unlink 21026 (time 1776919816.45 total 264.58 last 102.46) - open/close/unlink 22015 (time 1776919826.46 total 274.59 last 98.81) - open/close/unlink 23016 (time 1776919836.47 total 284.60 last 99.99) - open/close/unlink 23887 (time 1776919846.47 total 294.60 last 87.09) - open/close/unlink 24908 (time 1776919856.48 total 304.61 last 102.06) - open/close/unlink 25897 (time 1776919866.48 total 314.61 last 98.85) - open/close/unlink 26877 (time 1776919876.49 total 324.62 last 97.88) - open/close/unlink 27832 (time 1776919886.54 total 334.67 last 95.04) - open/close/unlink 28822 (time 1776919896.57 total 344.70 last 98.74) - open/close/unlink 29627 (time 1776919906.59 total 354.72 last 80.35) - open/close/unlink 30000 (time 1776919910.11 total 358.24 last 105.95) - open/close/unlink 30845 (time 1776919920.11 total 368.24 last 84.48) - open/close/unlink 31671 (time 1776919930.12 total 378.25 last 82.55) - open/close/unlink 32466 (time 1776919940.13 total 388.26 last 79.38) - open/close/unlink 33249 (time 1776919950.14 total 398.27 last 78.26) - open/close/unlink 34169 (time 1776919960.14 total 408.28 last 91.93) - open/close/unlink 35088 (time 1776919970.15 total 418.28 last 91.84) - open/close/unlink 35959 (time 1776919980.16 total 428.29 last 87.02) - open/close/unlink 36744 (time 1776919990.18 total 438.31 last 78.38) - open/close/unlink 37581 (time 1776920000.18 total 448.31 last 83.69) - open/close/unlink 38340 (time 1776920010.19 total 458.32 last 75.83) - open/close/unlink 39160 (time 1776920020.19 total 468.32 last 81.99) - open/close/unlink 40000 (time 1776920030.10 total 478.23 last 84.73) - open/close/unlink 40808 (time 1776920040.10 total 488.24 last 80.80) - open/close/unlink 41735 (time 1776920050.11 total 498.24 last 92.64) - open/close/unlink 42727 (time 1776920060.11 total 508.25 last 99.15) - open/close/unlink 43673 (time 1776920070.12 total 518.25 last 94.58) - open/close/unlink 44483 (time 1776920080.12 total 528.25 last 80.96) - open/close/unlink 45234 (time 1776920090.13 total 538.26 last 75.06) - open/close/unlink 46042 (time 1776920100.13 total 548.26 last 80.79) - open/close/unlink 46930 (time 1776920110.14 total 558.27 last 88.71) - open/cl - open/close/unlink 704 (time 1776919561.89 total 10.00 last 70.37) - open/close/unlink 1355 (time 1776919571.90 total 20.02 last 65.01) - open/close/unlink 2034 (time 1776919581.92 total 30.04 last 67.79) - open/close/unlink 2717 (time 1776919591.92 total 40.04 last 68.29) - open/close/unlink 3469 (time 1776919601.94 total 50.06 last 75.06) - open/close/unlink 4212 (time 1776919611.95 total 60.07 last 74.20) - open/close/unlink 4934 (time 1776919621.96 total 70.08 last 72.16) - open/close/unlink 5623 (time 1776919631.99 total 80.10 last 68.72) - open/close/unlink 6207 (time 1776919642.00 total 90.12 last 58.30) - open/close/unlink 6729 (time 1776919652.01 total 100.13 last 52.16) - open/close/unlink 7373 (time 1776919662.02 total 110.14 last 64.32) - open/close/unlink 8046 (time 1776919672.03 total 120.14 last 67.26) - open/close/unlink 8753 (time 1776919682.03 total 130.15 last 70.66) - open/close/unlink 9435 (time 1776919692.04 total 140.16 last 68.16) - open/close/unlink 10000 (time 1776919701.71 total 149.82 last 58.43) - open/close/unlink 10929 (time 1776919711.72 total 159.84 last 92.81) - open/close/unlink 11867 (time 1776919721.73 total 169.84 last 93.74) - open/close/unlink 12641 (time 1776919731.73 total 179.84 last 77.39) - open/close/unlink 13698 (time 1776919741.73 total 189.84 last 105.70) - open/close/unlink 14578 (time 1776919751.74 total 199.85 last 87.91) - open/close/unlink 15501 (time 1776919761.74 total 209.85 last 92.28) - open/close/unlink 16427 (time 1776919771.74 total 219.86 last 92.56) - open/close/unlink 17368 (time 1776919781.75 total 229.86 last 94.05) - open/close/unlink 18322 (time 1776919791.76 total 239.87 last 95.33) - open/close/unlink 19519 (time 1776919801.76 total 249.88 last 119.64) - open/close/unlink 20000 (time 1776919806.30 total 254.42 last 105.91) - open/close/unlink 21030 (time 1776919816.31 total 264.43 last 102.90) - open/close/unlink 22016 (time 1776919826.32 total 274.44 last 98.53) - open/close/unlink 23015 (time 1776919836.32 total 284.44 last 99.86) - open/close/unlink 23887 (time 1776919846.33 total 294.44 last 87.16) - open/close/unlink 24899 (time 1776919856.34 total 304.45 last 101.13) - open/close/unlink 25889 (time 1776919866.34 total 314.46 last 98.93) - open/close/unlink 26874 (time 1776919876.35 total 324.47 last 98.43) - open/close/unlink 27823 (time 1776919886.35 total 334.47 last 94.86) - open/close/unlink 28812 (time 1776919896.36 total 344.48 last 98.81) - open/close/unlink 29609 (time 1776919906.37 total 354.48 last 79.65) - open/close/unlink 30000 (time 1776919910.06 total 358.17 last 106.06) - open/close/unlink 30840 (time 1776919920.07 total 368.18 last 83.92) - open/close/unlink 31664 (time 1776919930.07 total 378.19 last 82.33) - open/close/unlink 32459 (time 1776919940.08 total 388.19 last 79.49) - open/close/unlink 33241 (time 1776919950.08 total 398.19 last 78.19) - open/close/unlink 34163 (time 1776919960.08 total 408.19 last 92.20) - open/close/unlink 35070 (time 1776919970.08 total 418.20 last 90.66) - open/close/unlink 35947 (time 1776919980.09 total 428.20 last 87.65) - open/close/unlink 36733 (time 1776919990.09 total 438.20 last 78.60) - open/close/unlink 37567 (time 1776920000.10 total 448.21 last 83.34) - open/close/unlink 38323 (time 1776920010.10 total 458.21 last 75.57) - open/close/unlink 39142 (time 1776920020.10 total 468.21 last 81.89) - open/close/unlink 39980 (time 1776920030.10 total 478.22 last 83.76) - open/close/unlink 40788 (time 1776920040.11 total 488.23 last 80.72) - open/close/unlink 41718 (time 1776920050.12 total 498.23 last 92.95) - open/close/unlink 42711 (time 1776920060.13 total 508.24 last 99.24) - open/close/unlink 43651 (time 1776920070.13 total 518.25 last 93.96) - open/close/unlink 44466 (time 1776920080.14 total 528.25 last 81.46) - open/close/unlink 45225 (time 1776920090.14 total 538.26 last 75.86) - open/close/unlink 46029 (time 1776920100.15 total 548.26 last 80.33) - open/close/unlink 46919 (time 1776920110.15 total 558.27 last 88.99) - open/cl - open/close/unlink 707 (time 1776919561.71 total 10.00 last 70.69) - open/close/unlink 1355 (time 1776919571.71 total 20.00 last 64.80) - open/close/unlink 2036 (time 1776919581.72 total 30.01 last 68.04) - open/close/unlink 2718 (time 1776919591.72 total 40.02 last 68.17) - open/close/unlink 3466 (time 1776919601.72 total 50.02 last 74.79) - open/close/unlink 4196 (time 1776919611.73 total 60.03 last 72.94) - open/close/unlink 4921 (time 1776919621.74 total 70.04 last 72.43) - open/close/unlink 5602 (time 1776919631.75 total 80.04 last 68.04) - open/close/unlink 6185 (time 1776919641.77 total 90.06 last 58.20) - open/close/unlink 6704 (time 1776919651.78 total 100.07 last 51.85) - open/close/unlink 7352 (time 1776919661.78 total 110.07 last 64.80) - open/close/unlink 8023 (time 1776919671.78 total 120.08 last 67.08) - open/close/unlink 8713 (time 1776919681.79 total 130.09 last 68.93) - open/close/unlink 9403 (time 1776919691.80 total 140.09 last 68.94) - open/close/unlink 9996 (time 1776919701.81 total 150.10 last 59.25) - open/close/unlink 10921 (time 1776919711.81 total 160.11 last 92.43) - open/close/unlink 11846 (time 1776919721.82 total 170.11 last 92.47) - open/close/unlink 12619 (time 1776919731.82 total 180.11 last 77.30) - open/close/unlink 13685 (time 1776919741.82 total 190.12 last 106.54) - open/close/unlink 14560 (time 1776919751.84 total 200.13 last 87.37) - open/close/unlink 15486 (time 1776919761.84 total 210.14 last 92.57) - open/close/unlink 16415 (time 1776919771.84 total 220.14 last 92.89) - open/close/unlink 17348 (time 1776919781.85 total 230.15 last 93.18) - open/close/unlink 18310 (time 1776919791.85 total 240.15 last 96.20) - open/close/unlink 19496 (time 1776919801.86 total 250.16 last 118.53) - open/close/unlink 20000 (time 1776919806.55 total 254.84 last 107.55) - open/close/unlink 21025 (time 1776919816.55 total 264.85 last 102.44) - open/close/unlink 22016 (time 1776919826.56 total 274.86 last 99.03) - open/close/unlink 23008 (time 1776919836.56 total 284.86 last 99.18) - open/close/unlink 23878 (time 1776919846.57 total 294.86 last 86.94) - open/close/unlink 24895 (time 1776919856.57 total 304.87 last 101.66) - open/close/unlink 25883 (time 1776919866.57 total 314.87 last 98.78) - open/close/unlink 26872 (time 1776919876.58 total 324.88 last 98.81) - open/close/unlink 27813 (time 1776919886.59 total 334.88 last 94.05) - open/close/unlink 28798 (time 1776919896.60 total 344.90 last 98.36) - open/close/unlink 29604 (time 1776919906.61 total 354.90 last 80.57) - open/close/unlink 30000 (time 1776919910.33 total 358.63 last 106.23) - open/close/unlink 30840 (time 1776919920.34 total 368.63 last 83.97) - open/close/unlink 31654 (time 1776919930.35 total 378.64 last 81.34) - open/close/unlink 32458 (time 1776919940.36 total 388.65 last 80.29) - open/close/unlink 33243 (time 1776919950.36 total 398.66 last 78.49) - open/close/unlink 34164 (time 1776919960.37 total 408.66 last 92.05) - open/close/unlink 35076 (time 1776919970.38 total 418.67 last 91.10) - open/close/unlink 35938 (time 1776919980.38 total 428.68 last 86.17) - open/close/unlink 36717 (time 1776919990.38 total 438.68 last 77.89) - open/close/unlink 37555 (time 1776920000.39 total 448.69 last 83.72) - open/close/unlink 38314 (time 1776920010.39 total 458.69 last 75.87) - open/close/unlink 39137 (time 1776920020.40 total 468.70 last 82.23) - open/close/unlink 39986 (time 1776920030.40 total 478.70 last 84.88) - open/close/unlink 40790 (time 1776920040.41 total 488.71 last 80.36) - open/close/unlink 41724 (time 1776920050.41 total 498.71 last 93.39) - open/close/unlink 42705 (time 1776920060.42 total 508.72 last 98.01) - open/close/unlink 43654 (time 1776920070.42 total 518.72 last 94.87) - open/close/unlink 44468 (time 1776920080.43 total 528.72 last 81.37) - open/close/unlink 45223 (time 1776920090.43 total 538.73 last 75.46) - open/close/unlink 46043 (time 1776920100.44 total 548.73 last 81.95) - open/close/unlink 46925 (time 1776920110.44 total 558.74 last 88.17) - open/clo/close/unlink 47807 (time 1776920119.25 total 567.60 last 88.08) - open/close/unlink 48721 (time 1776920129.27 total 577.62 last 91.27) - open/close/unlink 49545 (time 1776920139.28 total 587.63 last 82.33) - open/close/unlink 50000 (time 1776920145.02 total 593.37 last 79.25) - open/close/unlink 50606 (time 1776920155.05 total 603.40 last 60.43) - open/close/unlink 50848 (time 1776920165.05 total 613.41 last 24.18) - open/close/unlink 51062 (time 1776920175.06 total 623.41 last 21.39) - open/close/unlink 51304 (time 1776920185.07 total 633.42 last 24.16) - open/close/unlink 51505 (time 1776920195.09 total 643.44 last 20.07) - open/close/unlink 51679 (time 1776920205.16 total 653.51 last 17.28) - open/close/unlink 51886 (time 1776920215.17 total 663.52 last 20.67) - open/close/unlink 52090 (time 1776920225.22 total 673.57 last 20.30) - open/close/unlink 52297 (time 1776920235.24 total 683.59 last 20.66) - open/close/unlink 52570 (time 1776920245.24 total 693.59 last 27.29) - open/close/unlink 52842 (time 1776920255.25 total 703.60 last 27.18) - open/close/unlink 53171 (time 1776920265.25 total 713.60 last 32.90) - open/close/unlink 53544 (time 1776920275.26 total 723.61 last 37.25) - open/close/unlink 54009 (time 1776920285.27 total 733.62 last 46.46) - open/close/unlink 54435 (time 1776920295.28 total 743.63 last 42.56) - open/close/unlink 54769 (time 1776920305.28 total 753.64 last 33.39) - open/close/unlink 55080 (time 1776920315.30 total 763.65 last 31.04) - open/close/unlink 55391 (time 1776920325.30 total 773.65 last 31.10) - open/close/unlink 55726 (time 1776920335.31 total 783.66 last 33.48) - open/close/unlink 56080 (time 1776920345.33 total 793.68 last 35.33) - open/close/unlink 56439 (time 1776920355.35 total 803.70 last 35.81) - open/close/unlink 56967 (time 1776920365.35 total 813.70 last 52.80) - open/close/unlink 58171 (time 1776920375.36 total 823.71 last 120.35) - open/close/unlink 59483 (time 1776920385.36 total 833.71 last 131.17) - open/close/unlink 60000 (time 1776920390.26 total 838.61 last 105.48) - open/close/unlink 61165 (time 1776920400.27 total 848.62 last 116.45) - open/close/unlink 62679 (time 1776920410.27 total 858.62 last 151.35) - open/close/unlink 64046 (time 1776920420.27 total 868.63 last 136.64) total: 64768 open/close/unlink in 874.77 seconds: 74.04 ops/second ose/unlink 47832 (time 1776920119.89 total 568.01 last 88.43) - open/close/unlink 48732 (time 1776920129.90 total 578.02 last 89.93) - open/close/unlink 49546 (time 1776920139.90 total 588.02 last 81.38) - open/close/unlink 50000 (time 1776920145.40 total 593.52 last 82.62) - open/close/unlink 50578 (time 1776920155.40 total 603.52 last 57.77) - open/close/unlink 50818 (time 1776920165.41 total 613.53 last 23.98) - open/close/unlink 51038 (time 1776920175.48 total 623.60 last 21.85) - open/close/unlink 51280 (time 1776920185.50 total 633.61 last 24.17) - open/close/unlink 51484 (time 1776920195.58 total 643.69 last 20.24) - open/close/unlink 51663 (time 1776920205.61 total 653.73 last 17.84) - open/close/unlink 51868 (time 1776920215.62 total 663.74 last 20.47) - open/close/unlink 52064 (time 1776920225.65 total 673.77 last 19.54) - open/close/unlink 52278 (time 1776920235.67 total 683.79 last 21.35) - open/close/unlink 52556 (time 1776920245.71 total 693.82 last 27.71) - open/close/unlink 52837 (time 1776920255.73 total 703.84 last 28.05) - open/close/unlink 53165 (time 1776920265.75 total 713.86 last 32.72) - open/close/unlink 53534 (time 1776920275.77 total 723.89 last 36.82) - open/close/unlink 54000 (time 1776920285.79 total 733.90 last 46.52) - open/close/unlink 54417 (time 1776920295.82 total 743.93 last 41.58) - open/close/unlink 54753 (time 1776920305.83 total 753.95 last 33.55) - open/close/unlink 55076 (time 1776920315.84 total 763.96 last 32.27) - open/close/unlink 55382 (time 1776920325.87 total 773.99 last 30.51) - open/close/unlink 55714 (time 1776920335.88 total 783.99 last 33.18) - open/close/unlink 56078 (time 1776920345.89 total 794.01 last 36.35) - open/close/unlink 56436 (time 1776920355.90 total 804.01 last 35.78) - open/close/unlink 57012 (time 1776920365.90 total 814.02 last 57.58) - open/close/unlink 58225 (time 1776920375.90 total 824.02 last 121.23) - open/close/unlink 59507 (time 1776920385.92 total 834.03 last 128.06) - open/close/unlink 60000 (time 1776920390.49 total 838.61 last 107.72) - open/close/unlink 61161 (time 1776920400.50 total 848.62 last 116.03) - open/close/unlink 62686 (time 1776920410.50 total 858.62 last 152.43) - open/close/unlink 64037 (time 1776920420.51 total 868.62 last 135.04) total: 64768 open/close/unlink in 875.48 seconds: 73.98 ops/second ose/unlink 47807 (time 1776920120.16 total 568.27 last 88.72) - open/close/unlink 48700 (time 1776920130.18 total 578.29 last 89.17) - open/close/unlink 49513 (time 1776920140.18 total 588.30 last 81.24) - open/close/unlink 50000 (time 1776920145.95 total 594.07 last 84.39) - open/close/unlink 50553 (time 1776920156.01 total 604.13 last 54.97) - open/close/unlink 50794 (time 1776920166.01 total 614.13 last 24.10) - open/close/unlink 51023 (time 1776920176.03 total 624.15 last 22.85) - open/close/unlink 51258 (time 1776920186.04 total 634.16 last 23.48) - open/close/unlink 51466 (time 1776920196.06 total 644.17 last 20.77) - open/close/unlink 51645 (time 1776920206.08 total 654.20 last 17.86) - open/close/unlink 51850 (time 1776920216.10 total 664.21 last 20.46) - open/close/unlink 52050 (time 1776920226.17 total 674.28 last 19.86) - open/close/unlink 52272 (time 1776920236.18 total 684.30 last 22.17) - open/close/unlink 52559 (time 1776920246.20 total 694.32 last 28.64) - open/close/unlink 52833 (time 1776920256.24 total 704.36 last 27.29) - open/close/unlink 53169 (time 1776920266.26 total 714.38 last 33.53) - open/close/unlink 53547 (time 1776920276.27 total 724.38 last 37.77) - open/close/unlink 53997 (time 1776920286.28 total 734.40 last 44.94) - open/close/unlink 54434 (time 1776920296.29 total 744.40 last 43.67) - open/close/unlink 54758 (time 1776920306.29 total 754.40 last 32.40) - open/close/unlink 55078 (time 1776920316.32 total 764.43 last 31.91) - open/close/unlink 55383 (time 1776920326.35 total 774.46 last 30.41) - open/close/unlink 55714 (time 1776920336.35 total 784.47 last 33.10) - open/close/unlink 56078 (time 1776920346.37 total 794.49 last 36.32) - open/close/unlink 56443 (time 1776920356.37 total 804.49 last 36.49) - open/close/unlink 57061 (time 1776920366.38 total 814.49 last 61.77) - open/close/unlink 58280 (time 1776920376.38 total 824.50 last 121.85) - open/close/unlink 59554 (time 1776920386.39 total 834.50 last 127.34) - open/close/unlink 60000 (time 1776920390.66 total 838.77 last 104.45) - open/close/unlink 61181 (time 1776920400.67 total 848.78 last 118.00) - open/close/unlink 62702 (time 1776920410.67 total 858.78 last 152.08) - open/close/unlink 64052 (time 1776920420.67 total 868.79 last 134.90) total: 64768 open/close/unlink in 875.48 seconds: 73.98 ops/second ose/unlink 47820 (time 1776920120.14 total 568.27 last 89.00) - open/close/unlink 48709 (time 1776920130.14 total 578.27 last 88.89) - open/close/unlink 49514 (time 1776920140.16 total 588.29 last 80.35) - open/close/unlink 50000 (time 1776920145.96 total 594.10 last 83.71) - open/close/unlink 50545 (time 1776920156.01 total 604.14 last 54.26) - open/close/unlink 50792 (time 1776920166.04 total 614.17 last 24.62) - open/close/unlink 51015 (time 1776920176.05 total 624.18 last 22.28) - open/close/unlink 51249 (time 1776920186.06 total 634.19 last 23.38) - open/close/unlink 51443 (time 1776920196.10 total 644.23 last 19.32) - open/close/unlink 51616 (time 1776920206.10 total 654.24 last 17.29) - open/close/unlink 51829 (time 1776920216.18 total 664.31 last 21.14) - open/close/unlink 52030 (time 1776920226.19 total 674.32 last 20.07) - open/close/unlink 52252 (time 1776920236.22 total 684.35 last 22.13) - open/close/unlink 52532 (time 1776920246.24 total 694.37 last 27.96) - open/close/unlink 52809 (time 1776920256.25 total 704.39 last 27.65) - open/close/unlink 53142 (time 1776920266.28 total 714.41 last 33.23) - open/close/unlink 53524 (time 1776920276.28 total 724.42 last 38.17) - open/close/unlink 53976 (time 1776920286.33 total 734.46 last 45.01) - open/close/unlink 54406 (time 1776920296.35 total 744.49 last 42.89) - open/close/unlink 54743 (time 1776920306.38 total 754.52 last 33.60) - open/close/unlink 55065 (time 1776920316.39 total 764.52 last 32.19) - open/close/unlink 55375 (time 1776920326.41 total 774.54 last 30.93) - open/close/unlink 55703 (time 1776920336.42 total 784.56 last 32.75) - open/close/unlink 56067 (time 1776920346.42 total 794.56 last 36.40) - open/close/unlink 56430 (time 1776920356.45 total 804.58 last 36.20) - open/close/unlink 57047 (time 1776920366.46 total 814.59 last 61.66) - open/close/unlink 58268 (time 1776920376.46 total 824.60 last 122.02) - open/close/unlink 59540 (time 1776920386.47 total 834.60 last 127.17) - open/close/unlink 60000 (time 1776920390.85 total 838.98 last 105.01) - open/close/unlink 61183 (time 1776920400.85 total 848.98 last 118.27) - open/close/unlink 62720 (time 1776920410.86 total 858.99 last 153.54) - open/close/unlink 64057 (time 1776920420.86 total 869.00 last 133.63) total: 64768 open/close/unlink in 875.84 seconds: 73.95 ops/second se/unlink 47814 (time 1776920120.45 total 568.74 last 88.86) - open/close/unlink 48702 (time 1776920130.45 total 578.74 last 88.80) - open/close/unlink 49509 (time 1776920140.45 total 588.75 last 80.64) - open/close/unlink 50000 (time 1776920146.14 total 594.44 last 86.35) - open/close/unlink 50532 (time 1776920156.14 total 604.44 last 53.20) - open/close/unlink 50761 (time 1776920166.17 total 614.47 last 22.83) - open/close/unlink 50984 (time 1776920176.19 total 624.49 last 22.25) - open/close/unlink 51224 (time 1776920186.20 total 634.50 last 23.98) - open/close/unlink 51431 (time 1776920196.26 total 644.55 last 20.58) - open/close/unlink 51603 (time 1776920206.32 total 654.62 last 17.09) - open/close/unlink 51804 (time 1776920216.32 total 664.62 last 20.10) - open/close/unlink 52002 (time 1776920226.40 total 674.70 last 19.64) - open/close/unlink 52227 (time 1776920236.42 total 684.71 last 22.47) - open/close/unlink 52507 (time 1776920246.44 total 694.73 last 27.95) - open/close/unlink 52778 (time 1776920256.46 total 704.76 last 27.04) - open/close/unlink 53112 (time 1776920266.48 total 714.78 last 33.33) - open/close/unlink 53496 (time 1776920276.50 total 724.80 last 38.31) - open/close/unlink 53939 (time 1776920286.54 total 734.84 last 44.12) - open/close/unlink 54377 (time 1776920296.59 total 744.89 last 43.59) - open/close/unlink 54704 (time 1776920306.63 total 754.92 last 32.59) - open/close/unlink 55030 (time 1776920316.65 total 764.94 last 32.53) - open/close/unlink 55318 (time 1776920326.69 total 774.99 last 28.68) - open/close/unlink 55659 (time 1776920336.74 total 785.04 last 33.93) - open/close/unlink 56020 (time 1776920346.75 total 795.05 last 36.06) - open/close/unlink 56371 (time 1776920356.78 total 805.07 last 35.02) - open/close/unlink 57015 (time 1776920366.78 total 815.07 last 64.40) - open/close/unlink 58237 (time 1776920376.78 total 825.07 last 122.20) - open/close/unlink 59500 (time 1776920386.78 total 835.07 last 126.29) - open/close/unlink 60000 (time 1776920391.65 total 839.94 last 102.69) - open/close/unlink 61204 (time 1776920401.65 total 849.95 last 120.35) - open/close/unlink 62765 (time 1776920411.65 total 859.95 last 156.09) - open/close/unlink 64061 (time 1776920421.66 total 869.95 last 129.52) total: 64768 open/close/unlink in 877.51 seconds: 73.81 ops/second Waiting for MDT destroys to complete oleg415-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg415-client: oleg415-server: ssh exited with exit code 1 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg415-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg415-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg415-server stop mds service on oleg415-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg415-server stop mds service on oleg415-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg415-server unloading modules via unload_modules_local on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing unload_modules_local oleg415-server: modules unloaded. pdsh@oleg415-client: oleg415-client: ssh exited with exit code 2 pdsh@oleg415-client: oleg415-server: ssh exited with exit code 2 pdsh@oleg415-client: oleg415-client: ssh exited with exit code 2 pdsh@oleg415-client: oleg415-server: ssh exited with exit code 2 PASS 106 (956s) Stopping clients: oleg415-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg415-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg415-client: oleg415-server: ssh exited with exit code 2 oleg415-server: oleg415-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg415-server' oleg415-server: oleg415-server.virtnet: executing load_modules_local oleg415-server: Loading modules from /home/green/git/lustre-release/lustre oleg415-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg415-server: detected 4 online CPUs by sysfs oleg415-server: MODOPTS_LIBCFS= oleg415-server: Force libcfs to create 2 CPU partitions oleg415-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg415-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg415-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 1122 sec ========== 01:01:33 (1776920493) === conf-sanity: start cleanup 01:01:33 (1776920493) === === conf-sanity: finish cleanup 01:01:35 (1776920495) ===