-----============= acceptance-small: conf-sanity ============----- Thu Jun 11 22:57:40 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg647-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball Stopping clients: oleg647-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg647-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg647-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg647-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg647-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg647-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg647-server oleg647-server: oleg647-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg647-server' oleg647-server: oleg647-server.virtnet: executing load_modules_local oleg647-server: Loading modules from /home/green/git/lustre-release/lustre oleg647-server: detected 4 online CPUs by sysfs oleg647-server: MODOPTS_LIBCFS= oleg647-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg647-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg647-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg647-server: oleg647-server.virtnet: executing set_default_debug -1 all pdsh@oleg647-client: oleg647-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg647-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg647-server: oleg647-server.virtnet: executing set_default_debug -1 all pdsh@oleg647-client: oleg647-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg647-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg647-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg647-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg647-server: oleg647-server.virtnet: executing set_default_debug -1 all pdsh@oleg647-client: oleg647-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg647-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg647-server: oleg647-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg647-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg647-server: oleg647-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg647-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg647-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg647-server stop mds service on oleg647-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg647-server stop mds service on oleg647-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg647-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 23:00:01 (1781233201) start mds service on oleg647-server Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg647-server: oleg647-server.virtnet: executing set_default_debug -1 all pdsh@oleg647-client: oleg647-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg647-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg647-server: oleg647-server.virtnet: executing set_default_debug -1 all pdsh@oleg647-client: oleg647-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg647-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg647-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg647-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg647-server: oleg647-server.virtnet: executing set_default_debug -1 all pdsh@oleg647-client: oleg647-server: ssh exited with exit code 1 Started lustre-OST0000 oleg647-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg647-client.virtnet: -o user_xattr,flock 192.168.206.147@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 79992 274 79718 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 79992 255 79737 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 50000 433 49567 1% /mnt/lustre[OST:0] filesystem_summary: 50096 529 49567 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close/unlink 544 (time 1781233253.84 total 10.01 last 54.35) - open/close/unlink 1083 (time 1781233263.85 total 20.01 last 53.88) - open/close/unlink 1701 (time 1781233273.85 total 30.02 last 61.75) - open/close/unlink 2342 (time 1781233283.86 total 40.03 last 64.04) - open/close/unlink 3158 (time 1781233293.87 total 50.03 last 81.58) - open/close/unlink 3959 (time 1781233303.87 total 60.04 last 80.08) - open/close/unlink 4791 (time 1781233313.87 total 70.04 last 83.19) - open/close/unlink 5446 (time 1781233323.88 total 80.05 last 65.44) - open/close/unlink 6073 (time 1781233333.89 total 90.06 last 62.63) - open/close/unlink 6654 (time 1781233343.90 total 100.07 last 58.04) - open/close/unlink 7218 (time 1781233353.92 total 110.08 last 56.32) - open/close/unlink 7814 (time 1781233363.92 total 120.08 last 59.60) - open/close/unlink 8563 (time 1781233373.92 total 130.08 last 74.88) - open/close/unlink 9142 (time 1781233383.92 total 140.09 last 57.87) - open/close/unlink 9641 (time 1781233393.93 total 150.10 last 49.86) - open/close/unlink 10000 (time 1781233400.55 total 156.71 last 54.28) - open/close/unlink 10561 (time 1781233410.55 total 166.71 last 56.08) - open/close/unlink 11255 (time 1781233420.58 total 176.74 last 69.20) - open/close/unlink 11931 (time 1781233430.63 total 186.79 last 67.28) - open/close/unlink 12385 (time 1781233440.65 total 196.82 last 45.27) - open/close/unlink 12877 (time 1781233450.66 total 206.83 last 49.17) - open/close/unlink 13443 (time 1781233460.66 total 216.83 last 56.60) - open/close/unlink 14050 (time 1781233470.69 total 226.86 last 60.52) - open/close/unlink 14664 (time 1781233480.70 total 236.87 last 61.32) - open/close/unlink 15419 (time 1781233490.71 total 246.87 last 75.49) - open/close/unlink 16050 (time 1781233500.71 total 256.87 last 63.08) - open/close/unlink 16724 (time 1781233510.71 total 266.88 last 67.38) - open/close/unlink 17350 (time 1781233520.72 total 276.89 last 62.54) - open/close/unlink 18024 (time 1781233530.73 total 286.89 last 67.37) - open/close/unlink 18855 (time 1781233540.73 total 296.90 last 83.03) - open/close/unlink 19986 (time 1781233550.74 total 306.90 last 113.06) - open/close/unlink 21124 (time 1781233560.74 total 316.91 last 113.75) - open/close/unlink 22192 (time 1781233570.75 total 326.92 last 106.72) - open/close/unlink 23230 (time 1781233580.76 total 336.93 last 103.70) - open/close/unlink 24042 (time 1781233590.77 total 346.93 last 81.13) - open/close/unlink 24684 (time 1781233600.77 total 356.94 last 64.17) - open/close/unlink 25411 (time 1781233610.78 total 366.95 last 72.63) - open/close/unlink 26194 (time 1781233620.79 total 376.95 last 78.27) - open/close/unlink 26968 (time 1781233630.80 total 386.96 last 77.30) - open/close/unlink 27892 (time 1781233640.80 total 396.97 last 92.39) - open/close/unlink 28791 (time 1781233650.80 total 406.97 last 89.87) - open/close/unlink 29670 (time 1781233660.80 total 416.97 last 87.89) - open/close/unlink 30000 (time 1781233664.56 total 420.73 last 87.82) - open/close/unlink 30896 (time 1781233674.57 total 430.73 last 89.55) - open/close/unlink 31725 (time 1781233684.57 total 440.73 last 82.89) - open/close/unlink 32693 (time 1781233694.57 total 450.74 last 96.77) - open/close/unlink 33623 (time 1781233704.58 total 460.74 last 92.96) - open/close/unlink 34535 (time 1781233714.58 total 470.75 last 91.16) - open/close/unlink 35392 (time 1781233724.59 total 480.75 last 85.66) - open/close/unlink 36358 (time 1781233734.59 total 490.76 last 96.53) - open/close/unlink 37268 (time 1781233744.60 total 500.77 last 90.94) - open/close/unlink 38231 (time 1781233754.60 total 510.77 last 96.26) - open/close/unlink 39260 (time 1781233764.61 total 520.77 last 102.87) - open/close/unlink 40000 (time 1781233772.09 total 528.26 last 98.88) - open/close/unlink 40963 (time 1781233782.10 total 538.27 last 96.21) - open/close/unlink 42052 (time 1781233792.11 total 548.27 last 108.83) - open/close/unlink 43031 (time 1781233802.12 total 558.28 last 97.81) - open/clo - open/close/unlink 541 (time 1781233254.12 total 10.01 last 54.06) - open/close/unlink 1076 (time 1781233264.13 total 20.02 last 53.42) - open/close/unlink 1689 (time 1781233274.14 total 30.03 last 61.28) - open/close/unlink 2347 (time 1781233284.15 total 40.03 last 65.75) - open/close/unlink 3166 (time 1781233294.16 total 50.05 last 81.81) - open/close/unlink 3963 (time 1781233304.16 total 60.05 last 79.65) - open/close/unlink 4801 (time 1781233314.19 total 70.08 last 83.53) - open/close/unlink 5453 (time 1781233324.21 total 80.10 last 65.09) - open/close/unlink 6084 (time 1781233334.22 total 90.11 last 63.02) - open/close/unlink 6666 (time 1781233344.22 total 100.11 last 58.20) - open/close/unlink 7233 (time 1781233354.24 total 110.13 last 56.61) - open/close/unlink 7831 (time 1781233364.25 total 120.14 last 59.76) - open/close/unlink 8567 (time 1781233374.25 total 130.14 last 73.56) - open/close/unlink 9150 (time 1781233384.26 total 140.15 last 58.24) - open/close/unlink 9655 (time 1781233394.26 total 150.15 last 50.49) - open/close/unlink 10000 (time 1781233400.67 total 156.56 last 53.88) - open/close/unlink 10555 (time 1781233410.68 total 166.56 last 55.45) - open/close/unlink 11235 (time 1781233420.69 total 176.58 last 67.91) - open/close/unlink 11906 (time 1781233430.69 total 186.58 last 67.07) - open/close/unlink 12361 (time 1781233440.70 total 196.59 last 45.46) - open/close/unlink 12858 (time 1781233450.71 total 206.60 last 49.67) - open/close/unlink 13432 (time 1781233460.72 total 216.61 last 57.33) - open/close/unlink 14052 (time 1781233470.74 total 226.63 last 61.88) - open/close/unlink 14660 (time 1781233480.76 total 236.65 last 60.65) - open/close/unlink 15407 (time 1781233490.78 total 246.67 last 74.60) - open/close/unlink 16045 (time 1781233500.79 total 256.68 last 63.75) - open/close/unlink 16720 (time 1781233510.80 total 266.69 last 67.40) - open/close/unlink 17357 (time 1781233520.81 total 276.70 last 63.61) - open/close/unlink 18036 (time 1781233530.82 total 286.71 last 67.88) - open/close/unlink 18864 (time 1781233540.82 total 296.71 last 82.75) - open/close/unlink 19998 (time 1781233550.82 total 306.71 last 113.39) - open/close/unlink 21135 (time 1781233560.83 total 316.72 last 113.63) - open/close/unlink 22199 (time 1781233570.84 total 326.73 last 106.31) - open/close/unlink 23233 (time 1781233580.85 total 336.73 last 103.31) - open/close/unlink 24042 (time 1781233590.86 total 346.74 last 80.82) - open/close/unlink 24688 (time 1781233600.86 total 356.75 last 64.55) - open/close/unlink 25413 (time 1781233610.87 total 366.76 last 72.46) - open/close/unlink 26197 (time 1781233620.88 total 376.77 last 78.34) - open/close/unlink 26955 (time 1781233630.89 total 386.78 last 75.73) - open/close/unlink 27880 (time 1781233640.89 total 396.78 last 92.45) - open/close/unlink 28778 (time 1781233650.89 total 406.78 last 89.79) - open/close/unlink 29648 (time 1781233660.90 total 416.79 last 86.95) - open/close/unlink 30000 (time 1781233664.90 total 420.79 last 87.99) - open/close/unlink 30901 (time 1781233674.92 total 430.81 last 89.92) - open/close/unlink 31739 (time 1781233684.92 total 440.81 last 83.77) - open/close/unlink 32707 (time 1781233694.93 total 450.82 last 96.68) - open/close/unlink 33632 (time 1781233704.95 total 460.83 last 92.40) - open/close/unlink 34553 (time 1781233714.95 total 470.84 last 92.09) - open/close/unlink 35410 (time 1781233724.96 total 480.85 last 85.62) - open/close/unlink 36365 (time 1781233734.96 total 490.85 last 95.43) - open/close/unlink 37283 (time 1781233744.97 total 500.85 last 91.78) - open/close/unlink 38252 (time 1781233754.97 total 510.86 last 96.85) - open/close/unlink 39265 (time 1781233764.98 total 520.87 last 101.24) - open/close/unlink 40000 (time 1781233772.35 total 528.23 last 99.75) - open/close/unlink 40951 (time 1781233782.35 total 538.23 last 95.10) - open/close/unlink 42033 (time 1781233792.35 total 548.24 last 108.17) - open/close/unlink 43025 (time 1781233802.36 total 558.24 last 99.13) - open/clo - open/close/unlink 548 (time 1781233254.20 total 10.02 last 54.68) - open/close/unlink 1089 (time 1781233264.20 total 20.03 last 54.07) - open/close/unlink 1725 (time 1781233274.21 total 30.03 last 63.55) - open/close/unlink 2378 (time 1781233284.22 total 40.04 last 65.27) - open/close/unlink 3185 (time 1781233294.22 total 50.04 last 80.69) - open/close/unlink 3985 (time 1781233304.22 total 60.04 last 79.96) - open/close/unlink 4819 (time 1781233314.24 total 70.06 last 83.24) - open/close/unlink 5475 (time 1781233324.25 total 80.08 last 65.51) - open/close/unlink 6103 (time 1781233334.27 total 90.09 last 62.70) - open/close/unlink 6678 (time 1781233344.28 total 100.10 last 57.44) - open/close/unlink 7247 (time 1781233354.29 total 110.11 last 56.87) - open/close/unlink 7842 (time 1781233364.29 total 120.11 last 59.47) - open/close/unlink 8574 (time 1781233374.32 total 130.14 last 73.02) - open/close/unlink 9151 (time 1781233384.34 total 140.16 last 57.58) - open/close/unlink 9656 (time 1781233394.34 total 150.16 last 50.48) - open/close/unlink 10000 (time 1781233400.85 total 156.67 last 52.89) - open/close/unlink 10563 (time 1781233410.85 total 166.68 last 56.26) - open/close/unlink 11247 (time 1781233420.86 total 176.68 last 68.35) - open/close/unlink 11923 (time 1781233430.87 total 186.69 last 67.56) - open/close/unlink 12375 (time 1781233440.87 total 196.69 last 45.19) - open/close/unlink 12871 (time 1781233450.88 total 206.70 last 49.55) - open/close/unlink 13444 (time 1781233460.88 total 216.70 last 57.29) - open/close/unlink 14046 (time 1781233470.89 total 226.71 last 60.15) - open/close/unlink 14648 (time 1781233480.90 total 236.72 last 60.13) - open/close/unlink 15398 (time 1781233490.91 total 246.73 last 74.97) - open/close/unlink 16036 (time 1781233500.91 total 256.73 last 63.78) - open/close/unlink 16706 (time 1781233510.91 total 266.74 last 66.95) - open/close/unlink 17331 (time 1781233520.92 total 276.74 last 62.49) - open/close/unlink 18007 (time 1781233530.92 total 286.74 last 67.60) - open/close/unlink 18838 (time 1781233540.93 total 296.75 last 82.99) - open/close/unlink 19975 (time 1781233550.93 total 306.75 last 113.70) - open/close/unlink 21119 (time 1781233560.94 total 316.76 last 114.32) - open/close/unlink 22184 (time 1781233570.96 total 326.78 last 106.30) - open/close/unlink 23223 (time 1781233580.96 total 336.78 last 103.87) - open/close/unlink 24027 (time 1781233590.97 total 346.79 last 80.35) - open/close/unlink 24669 (time 1781233600.98 total 356.80 last 64.11) - open/close/unlink 25404 (time 1781233610.99 total 366.81 last 73.46) - open/close/unlink 26187 (time 1781233620.99 total 376.82 last 78.25) - open/close/unlink 26950 (time 1781233630.99 total 386.82 last 76.29) - open/close/unlink 27881 (time 1781233641.00 total 396.82 last 93.04) - open/close/unlink 28784 (time 1781233651.01 total 406.83 last 90.26) - open/close/unlink 29654 (time 1781233661.01 total 416.83 last 86.98) - open/close/unlink 30000 (time 1781233664.94 total 420.76 last 88.03) - open/close/unlink 30896 (time 1781233674.94 total 430.77 last 89.56) - open/close/unlink 31724 (time 1781233684.95 total 440.77 last 82.76) - open/close/unlink 32686 (time 1781233694.95 total 450.77 last 96.18) - open/close/unlink 33613 (time 1781233704.95 total 460.77 last 92.69) - open/close/unlink 34536 (time 1781233714.95 total 470.77 last 92.28) - open/close/unlink 35383 (time 1781233724.96 total 480.78 last 84.62) - open/close/unlink 36336 (time 1781233734.97 total 490.79 last 95.21) - open/close/unlink 37256 (time 1781233744.98 total 500.80 last 91.93) - open/close/unlink 38227 (time 1781233754.99 total 510.81 last 97.03) - open/close/unlink 39237 (time 1781233764.99 total 520.81 last 100.98) - open/close/unlink 40000 (time 1781233772.73 total 528.55 last 98.59) - open/close/unlink 40965 (time 1781233782.73 total 538.55 last 96.45) - open/close/unlink 42050 (time 1781233792.73 total 548.56 last 108.46) - open/close/unlink 43046 (time 1781233802.73 total 558.56 last 99.60) - open/clo - open/close/unlink 548 (time 1781233254.17 total 10.00 last 54.77) - open/close/unlink 1086 (time 1781233264.17 total 20.01 last 53.79) - open/close/unlink 1708 (time 1781233274.18 total 30.02 last 62.13) - open/close/unlink 2365 (time 1781233284.18 total 40.02 last 65.69) - open/close/unlink 3183 (time 1781233294.19 total 50.03 last 81.75) - open/close/unlink 3978 (time 1781233304.19 total 60.03 last 79.48) - open/close/unlink 4806 (time 1781233314.21 total 70.05 last 82.67) - open/close/unlink 5454 (time 1781233324.22 total 80.06 last 64.72) - open/close/unlink 6090 (time 1781233334.24 total 90.07 last 63.50) - open/close/unlink 6675 (time 1781233344.27 total 100.10 last 58.34) - open/close/unlink 7236 (time 1781233354.30 total 110.14 last 55.90) - open/close/unlink 7844 (time 1781233364.31 total 120.15 last 60.75) - open/close/unlink 8585 (time 1781233374.32 total 130.15 last 74.06) - open/close/unlink 9160 (time 1781233384.34 total 140.17 last 57.37) - open/close/unlink 9668 (time 1781233394.35 total 150.18 last 50.74) - open/close/unlink 10000 (time 1781233400.63 total 156.47 last 52.85) - open/close/unlink 10559 (time 1781233410.65 total 166.49 last 55.79) - open/close/unlink 11239 (time 1781233420.67 total 176.51 last 67.88) - open/close/unlink 11911 (time 1781233430.67 total 186.51 last 67.16) - open/close/unlink 12357 (time 1781233440.69 total 196.53 last 44.52) - open/close/unlink 12848 (time 1781233450.71 total 206.55 last 49.02) - open/close/unlink 13422 (time 1781233460.74 total 216.57 last 57.25) - open/close/unlink 14042 (time 1781233470.76 total 226.59 last 61.87) - open/close/unlink 14644 (time 1781233480.76 total 236.60 last 60.15) - open/close/unlink 15386 (time 1781233490.77 total 246.60 last 74.18) - open/close/unlink 16027 (time 1781233500.78 total 256.61 last 64.05) - open/close/unlink 16700 (time 1781233510.79 total 266.63 last 67.20) - open/close/unlink 17327 (time 1781233520.79 total 276.63 last 62.67) - open/close/unlink 18001 (time 1781233530.81 total 286.64 last 67.32) - open/close/unlink 18824 (time 1781233540.81 total 296.65 last 82.27) - open/close/unlink 19967 (time 1781233550.82 total 306.65 last 114.22) - open/close/unlink 21108 (time 1781233560.82 total 316.66 last 114.06) - open/close/unlink 22169 (time 1781233570.82 total 326.66 last 106.04) - open/close/unlink 23200 (time 1781233580.83 total 336.67 last 103.04) - open/close/unlink 24018 (time 1781233590.83 total 346.67 last 81.78) - open/close/unlink 24659 (time 1781233600.84 total 356.68 last 64.04) - open/close/unlink 25383 (time 1781233610.84 total 366.68 last 72.38) - open/close/unlink 26166 (time 1781233620.85 total 376.68 last 78.27) - open/close/unlink 26929 (time 1781233630.86 total 386.70 last 76.19) - open/close/unlink 27847 (time 1781233640.86 total 396.70 last 91.79) - open/close/unlink 28740 (time 1781233650.86 total 406.70 last 89.30) - open/close/unlink 29610 (time 1781233660.87 total 416.70 last 86.98) - open/close/unlink 30000 (time 1781233665.20 total 421.03 last 90.09) - open/close/unlink 30882 (time 1781233675.22 total 431.05 last 88.03) - open/close/unlink 31727 (time 1781233685.22 total 441.05 last 84.49) - open/close/unlink 32693 (time 1781233695.22 total 451.06 last 96.57) - open/close/unlink 33623 (time 1781233705.22 total 461.06 last 92.98) - open/close/unlink 34542 (time 1781233715.23 total 471.06 last 91.84) - open/close/unlink 35397 (time 1781233725.24 total 481.07 last 85.43) - open/close/unlink 36350 (time 1781233735.24 total 491.07 last 95.28) - open/close/unlink 37274 (time 1781233745.25 total 501.08 last 92.32) - open/close/unlink 38245 (time 1781233755.25 total 511.09 last 97.05) - open/close/unlink 39261 (time 1781233765.26 total 521.09 last 101.57) - open/close/unlink 40000 (time 1781233772.82 total 528.66 last 97.69) - open/close/unlink 40973 (time 1781233782.82 total 538.66 last 97.28) - open/close/unlink 42049 (time 1781233792.83 total 548.66 last 107.56) - open/close/unlink 43047 (time 1781233802.83 total 558.67 last 99.72) - open/clo - open/close/unlink 547 (time 1781233253.90 total 10.02 last 54.61) - open/close/unlink 1086 (time 1781233263.90 total 20.02 last 53.89) - open/close/unlink 1705 (time 1781233273.91 total 30.03 last 61.84) - open/close/unlink 2350 (time 1781233283.92 total 40.04 last 64.46) - open/close/unlink 3162 (time 1781233293.93 total 50.04 last 81.14) - open/close/unlink 3962 (time 1781233303.93 total 60.04 last 79.99) - open/close/unlink 4797 (time 1781233313.93 total 70.04 last 83.49) - open/close/unlink 5450 (time 1781233323.95 total 80.06 last 65.18) - open/close/unlink 6082 (time 1781233333.95 total 90.07 last 63.16) - open/close/unlink 6664 (time 1781233343.96 total 100.08 last 58.16) - open/close/unlink 7227 (time 1781233353.98 total 110.09 last 56.21) - open/close/unlink 7816 (time 1781233363.98 total 120.10 last 58.87) - open/close/unlink 8571 (time 1781233373.99 total 130.11 last 75.40) - open/close/unlink 9141 (time 1781233384.01 total 140.12 last 56.93) - open/close/unlink 9634 (time 1781233394.01 total 150.13 last 49.27) - open/close/unlink 10000 (time 1781233400.79 total 156.90 last 54.03) - open/close/unlink 10565 (time 1781233410.80 total 166.91 last 56.45) - open/close/unlink 11242 (time 1781233420.81 total 176.93 last 67.59) - open/close/unlink 11910 (time 1781233430.83 total 186.94 last 66.71) - open/close/unlink 12366 (time 1781233440.84 total 196.96 last 45.54) - open/close/unlink 12864 (time 1781233450.85 total 206.96 last 49.76) - open/close/unlink 13434 (time 1781233460.85 total 216.96 last 57.00) - open/close/unlink 14048 (time 1781233470.85 total 226.97 last 61.37) - open/close/unlink 14656 (time 1781233480.86 total 236.97 last 60.77) - open/close/unlink 15411 (time 1781233490.87 total 246.99 last 75.41) - open/close/unlink 16041 (time 1781233500.88 total 257.00 last 62.93) - open/close/unlink 16711 (time 1781233510.89 total 267.01 last 66.95) - open/close/unlink 17349 (time 1781233520.93 total 277.05 last 63.54) - open/close/unlink 18024 (time 1781233530.96 total 287.07 last 67.31) - open/close/unlink 18857 (time 1781233540.96 total 297.08 last 83.26) - open/close/unlink 19982 (time 1781233550.97 total 307.08 last 112.46) - open/close/unlink 21117 (time 1781233560.98 total 317.09 last 113.38) - open/close/unlink 22174 (time 1781233570.98 total 327.09 last 105.69) - open/close/unlink 23204 (time 1781233580.98 total 337.10 last 102.98) - open/close/unlink 24018 (time 1781233590.99 total 347.10 last 81.36) - open/close/unlink 24652 (time 1781233600.99 total 357.10 last 63.38) - open/close/unlink 25378 (time 1781233610.99 total 367.11 last 72.57) - open/close/unlink 26164 (time 1781233621.01 total 377.12 last 78.50) - open/close/unlink 26922 (time 1781233631.01 total 387.13 last 75.74) - open/close/unlink 27852 (time 1781233641.03 total 397.14 last 92.89) - open/close/unlink 28745 (time 1781233651.04 total 407.15 last 89.22) - open/close/unlink 29599 (time 1781233661.04 total 417.16 last 85.32) - open/close/unlink 30000 (time 1781233665.53 total 421.65 last 89.32) - open/close/unlink 30884 (time 1781233675.54 total 431.66 last 88.34) - open/close/unlink 31734 (time 1781233685.55 total 441.67 last 84.91) - open/close/unlink 32707 (time 1781233695.56 total 451.67 last 97.24) - open/close/unlink 33633 (time 1781233705.57 total 461.68 last 92.52) - open/close/unlink 34552 (time 1781233715.57 total 471.68 last 91.89) - open/close/unlink 35415 (time 1781233725.57 total 481.69 last 86.26) - open/close/unlink 36364 (time 1781233735.59 total 491.70 last 94.78) - open/close/unlink 37286 (time 1781233745.59 total 501.70 last 92.20) - open/close/unlink 38250 (time 1781233755.60 total 511.71 last 96.32) - open/close/unlink 39257 (time 1781233765.61 total 521.72 last 100.59) - open/close/unlink 40000 (time 1781233773.08 total 529.20 last 99.43) - open/close/unlink 40965 (time 1781233783.09 total 539.20 last 96.43) - open/close/unlink 42052 (time 1781233793.09 total 549.21 last 108.66) - open/close/unlink 43048 (time 1781233803.11 total 559.23 last 99.41) - open/close/unlink 43865 (time 1781233812.84 total 568.68 last 81.75) - open/close/unlink 44658 (time 1781233822.85 total 578.69 last 79.22) - open/close/unlink 45440 (time 1781233832.86 total 588.70 last 78.12) - open/close/unlink 46159 (time 1781233842.86 total 598.70 last 71.87) - open/close/unlink 47045 (time 1781233852.87 total 608.70 last 88.60) - open/close/unlink 47662 (time 1781233862.87 total 618.70 last 61.68) - open/close/unlink 48466 (time 1781233872.87 total 628.71 last 80.39) - open/close/unlink 49217 (time 1781233882.88 total 638.72 last 75.03) - open/close/unlink 50000 (time 1781233892.58 total 648.41 last 80.74) - open/close/unlink 50723 (time 1781233902.58 total 658.42 last 72.25) - open/close/unlink 51665 (time 1781233912.59 total 668.42 last 94.16) - open/close/unlink 52447 (time 1781233922.59 total 678.43 last 78.18) - open/close/unlink 53229 (time 1781233932.59 total 688.43 last 78.19) - open/close/unlink 53979 (time 1781233942.60 total 698.44 last 74.92) - open/close/unlink 54667 (time 1781233952.61 total 708.45 last 68.72) - open/close/unlink 55484 (time 1781233962.62 total 718.46 last 81.64) - open/close/unlink 56397 (time 1781233972.62 total 728.46 last 91.29) - open/close/unlink 57171 (time 1781233982.63 total 738.46 last 77.38) - open/close/unlink 58040 (time 1781233992.63 total 748.47 last 86.82) - open/close/unlink 58843 (time 1781234002.64 total 758.48 last 80.22) - open/close/unlink 59437 (time 1781234012.67 total 768.50 last 59.27) - open/close/unlink 60000 (time 1781234021.77 total 777.60 last 61.86) - open/close/unlink 60739 (time 1781234031.77 total 787.61 last 73.86) - open/close/unlink 61509 (time 1781234041.77 total 797.61 last 76.99) - open/close/unlink 62457 (time 1781234051.77 total 807.61 last 94.77) - open/close/unlink 63429 (time 1781234061.78 total 817.61 last 97.16) - open/close/unlink 64412 (time 1781234071.78 total 827.62 last 98.29) total: 64768 open/close/unlink in 830.83 seconds: 77.96 ops/second se/unlink 43833 (time 1781233812.36 total 568.25 last 80.78) - open/close/unlink 44626 (time 1781233822.36 total 578.25 last 79.26) - open/close/unlink 45399 (time 1781233832.37 total 588.26 last 77.24) - open/close/unlink 46107 (time 1781233842.37 total 598.26 last 70.80) - open/close/unlink 46991 (time 1781233852.37 total 608.26 last 88.39) - open/close/unlink 47611 (time 1781233862.39 total 618.27 last 61.92) - open/close/unlink 48408 (time 1781233872.39 total 628.28 last 79.63) - open/close/unlink 49174 (time 1781233882.40 total 638.28 last 76.59) - open/close/unlink 49973 (time 1781233892.40 total 648.29 last 79.85) - open/close/unlink 50686 (time 1781233902.41 total 658.30 last 71.23) - open/close/unlink 51633 (time 1781233912.43 total 668.32 last 94.53) - open/close/unlink 52418 (time 1781233922.43 total 678.32 last 78.47) - open/close/unlink 53210 (time 1781233932.44 total 688.32 last 79.19) - open/close/unlink 53963 (time 1781233942.45 total 698.34 last 75.21) - open/close/unlink 54653 (time 1781233952.46 total 708.35 last 68.93) - open/close/unlink 55463 (time 1781233962.46 total 718.35 last 80.99) - open/close/unlink 56384 (time 1781233972.47 total 728.35 last 92.04) - open/close/unlink 57160 (time 1781233982.47 total 738.36 last 77.55) - open/close/unlink 58027 (time 1781233992.47 total 748.36 last 86.69) - open/close/unlink 58838 (time 1781234002.49 total 758.38 last 80.94) - open/close/unlink 59437 (time 1781234012.51 total 768.40 last 59.79) - open/close/unlink 60000 (time 1781234021.65 total 777.54 last 61.60) - open/close/unlink 60735 (time 1781234031.65 total 787.54 last 73.49) - open/close/unlink 61500 (time 1781234041.65 total 797.54 last 76.48) - open/close/unlink 62439 (time 1781234051.66 total 807.55 last 93.85) - open/close/unlink 63400 (time 1781234061.66 total 817.55 last 96.09) - open/close/unlink 64383 (time 1781234071.66 total 827.55 last 98.25) total: 64768 open/close/unlink in 831.13 seconds: 77.93 ops/second se/unlink 43847 (time 1781233812.13 total 568.29 last 81.51) - open/close/unlink 44636 (time 1781233822.13 total 578.30 last 78.86) - open/close/unlink 45405 (time 1781233832.15 total 588.31 last 76.77) - open/close/unlink 46121 (time 1781233842.15 total 598.32 last 71.57) - open/close/unlink 47003 (time 1781233852.16 total 608.32 last 88.17) - open/close/unlink 47621 (time 1781233862.18 total 618.35 last 61.66) - open/close/unlink 48417 (time 1781233872.19 total 628.36 last 79.52) - open/close/unlink 49179 (time 1781233882.19 total 638.36 last 76.19) - open/close/unlink 49961 (time 1781233892.19 total 648.36 last 78.19) - open/close/unlink 50688 (time 1781233902.19 total 658.36 last 72.68) - open/close/unlink 51626 (time 1781233912.21 total 668.37 last 93.67) - open/close/unlink 52404 (time 1781233922.21 total 678.37 last 77.79) - open/close/unlink 53198 (time 1781233932.22 total 688.38 last 79.31) - open/close/unlink 53946 (time 1781233942.22 total 698.39 last 74.78) - open/close/unlink 54634 (time 1781233952.24 total 708.40 last 68.70) - open/close/unlink 55432 (time 1781233962.24 total 718.40 last 79.80) - open/close/unlink 56351 (time 1781233972.24 total 728.41 last 91.85) - open/close/unlink 57126 (time 1781233982.25 total 738.41 last 77.45) - open/close/unlink 58000 (time 1781233992.25 total 748.42 last 87.36) - open/close/unlink 58825 (time 1781234002.26 total 758.42 last 82.45) - open/close/unlink 59424 (time 1781234012.27 total 768.44 last 59.82) - open/close/unlink 60000 (time 1781234021.69 total 777.85 last 61.16) - open/close/unlink 60729 (time 1781234031.69 total 787.86 last 72.87) - open/close/unlink 61496 (time 1781234041.70 total 797.86 last 76.66) - open/close/unlink 62435 (time 1781234051.70 total 807.87 last 93.84) - open/close/unlink 63408 (time 1781234061.71 total 817.87 last 97.26) - open/close/unlink 64386 (time 1781234071.72 total 827.88 last 97.70) total: 64768 open/close/unlink in 831.52 seconds: 77.89 ops/second se/unlink 43859 (time 1781233812.74 total 568.56 last 81.26) - open/close/unlink 44660 (time 1781233822.75 total 578.57 last 80.02) - open/close/unlink 45440 (time 1781233832.76 total 588.58 last 77.93) - open/close/unlink 46152 (time 1781233842.77 total 598.59 last 71.14) - open/close/unlink 47035 (time 1781233852.77 total 608.60 last 88.25) - open/close/unlink 47653 (time 1781233862.78 total 618.60 last 61.75) - open/close/unlink 48461 (time 1781233872.79 total 628.61 last 80.73) - open/close/unlink 49219 (time 1781233882.80 total 638.62 last 75.72) - open/close/unlink 50000 (time 1781233892.62 total 648.44 last 79.55) - open/close/unlink 50717 (time 1781233902.63 total 658.45 last 71.65) - open/close/unlink 51652 (time 1781233912.63 total 668.45 last 93.49) - open/close/unlink 52425 (time 1781233922.64 total 678.46 last 77.22) - open/close/unlink 53204 (time 1781233932.64 total 688.47 last 77.85) - open/close/unlink 53952 (time 1781233942.65 total 698.47 last 74.77) - open/close/unlink 54634 (time 1781233952.65 total 708.47 last 68.20) - open/close/unlink 55454 (time 1781233962.66 total 718.48 last 81.94) - open/close/unlink 56366 (time 1781233972.66 total 728.48 last 91.17) - open/close/unlink 57144 (time 1781233982.67 total 738.49 last 77.75) - open/close/unlink 58016 (time 1781233992.68 total 748.50 last 87.08) - open/close/unlink 58820 (time 1781234002.69 total 758.51 last 80.35) - open/close/unlink 59412 (time 1781234012.70 total 768.53 last 59.10) - open/close/unlink 60000 (time 1781234022.32 total 778.14 last 61.14) - open/close/unlink 60756 (time 1781234032.34 total 788.16 last 75.45) - open/close/unlink 61528 (time 1781234042.35 total 798.17 last 77.16) - open/close/unlink 62489 (time 1781234052.35 total 808.17 last 96.07) - open/close/unlink 63451 (time 1781234062.35 total 818.17 last 96.18) - open/close/unlink 64432 (time 1781234072.36 total 828.18 last 98.00) total: 64768 open/close/unlink in 831.56 seconds: 77.89 ops/second se/unlink 43875 (time 1781233813.13 total 569.24 last 82.56) - open/close/unlink 44662 (time 1781233823.14 total 579.25 last 78.62) - open/close/unlink 45440 (time 1781233833.15 total 589.26 last 77.71) - open/close/unlink 46165 (time 1781233843.15 total 599.27 last 72.48) - open/close/unlink 47049 (time 1781233853.16 total 609.28 last 88.33) - open/close/unlink 47661 (time 1781233863.17 total 619.28 last 61.17) - open/close/unlink 48462 (time 1781233873.17 total 629.29 last 80.06) - open/close/unlink 49216 (time 1781233883.17 total 639.29 last 75.39) - open/close/unlink 50000 (time 1781233892.92 total 649.03 last 80.45) - open/close/unlink 50713 (time 1781233902.92 total 659.04 last 71.27) - open/close/unlink 51643 (time 1781233912.92 total 669.04 last 92.99) - open/close/unlink 52423 (time 1781233922.95 total 679.06 last 77.82) - open/close/unlink 53199 (time 1781233932.95 total 689.07 last 77.53) - open/close/unlink 53953 (time 1781233942.96 total 699.07 last 75.37) - open/close/unlink 54648 (time 1781233952.97 total 709.09 last 69.41) - open/close/unlink 55463 (time 1781233962.98 total 719.09 last 81.45) - open/close/unlink 56381 (time 1781233972.98 total 729.10 last 91.76) - open/close/unlink 57153 (time 1781233982.98 total 739.10 last 77.18) - open/close/unlink 58006 (time 1781233992.98 total 749.10 last 85.29) - open/close/unlink 58801 (time 1781234003.00 total 759.12 last 79.37) - open/close/unlink 59408 (time 1781234013.01 total 769.12 last 60.67) - open/close/unlink 60000 (time 1781234022.67 total 778.79 last 61.24) - open/close/unlink 60758 (time 1781234032.69 total 788.80 last 75.71) - open/close/unlink 61531 (time 1781234042.69 total 798.80 last 77.28) - open/close/unlink 62497 (time 1781234052.69 total 808.81 last 96.54) - open/close/unlink 63463 (time 1781234062.70 total 818.81 last 96.56) - open/close/unlink 64443 (time 1781234072.70 total 828.81 last 97.99) total: 64768 open/close/unlink in 832.31 seconds: 77.82 ops/second Waiting for MDT destroys to complete oleg647-server: fstrim: /mnt/lustre-ost2: the discard operation is not supported pdsh@oleg647-client: oleg647-server: ssh exited with exit code 1 fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg647-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg647-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg647-server stop mds service on oleg647-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg647-server stop mds service on oleg647-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg647-server unloading modules via unload_modules_local on: 'oleg647-server' oleg647-server: oleg647-server.virtnet: executing unload_modules_local oleg647-server: modules unloaded. pdsh@oleg647-client: oleg647-client: ssh exited with exit code 2 pdsh@oleg647-client: oleg647-server: ssh exited with exit code 2 pdsh@oleg647-client: oleg647-client: ssh exited with exit code 2 pdsh@oleg647-client: oleg647-server: ssh exited with exit code 2 PASS 106 (923s) Stopping clients: oleg647-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg647-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg647-client: oleg647-server: ssh exited with exit code 2 oleg647-server: oleg647-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg647-server' oleg647-server: oleg647-server.virtnet: executing load_modules_local oleg647-server: Loading modules from /home/green/git/lustre-release/lustre oleg647-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg647-server: detected 4 online CPUs by sysfs oleg647-server: MODOPTS_LIBCFS= oleg647-server: Force libcfs to create 2 CPU partitions oleg647-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg647-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg647-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 1097 sec ========== 23:15:57 (1781234157) === conf-sanity: start cleanup 23:15:58 (1781234158) === === conf-sanity: finish cleanup 23:15:59 (1781234159) ===