-----============= acceptance-small: conf-sanity ============----- Thu Mar 19 13:29:09 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg260-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg260-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg260-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg260-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg260-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg260-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg260-server oleg260-server: oleg260-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: MODOPTS_LIBCFS= oleg260-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg260-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg260-server: oleg260-server.virtnet: executing set_default_debug -1 all pdsh@oleg260-client: oleg260-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg260-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg260-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg260-server: oleg260-server.virtnet: executing set_default_debug -1 all pdsh@oleg260-client: oleg260-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg260-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg260-server: oleg260-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg260-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg260-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg260-server stop mds service on oleg260-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg260-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 13:30:32 (1773941432) Stopping clients: oleg260-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg260-client.virtnet /mnt/lustre2 (opts:-f) oleg260-server: oleg260-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: MODOPTS_LIBCFS= oleg260-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg260-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg260-server: oleg260-server.virtnet: executing set_default_debug -1 all pdsh@oleg260-client: oleg260-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg260-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg260-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg260-server: oleg260-server.virtnet: executing set_default_debug -1 all pdsh@oleg260-client: oleg260-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg260-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg260-client.virtnet: -o user_xattr,flock 192.168.202.160@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 4482 (time 1773941477.47 total 10.00 last 448.15) - open/close 8884 (time 1773941487.47 total 20.00 last 440.12) - open/close 10000 (time 1773941490.35 total 22.88 last 387.31) - open/close 14573 (time 1773941500.35 total 32.89 last 457.26) - open/close 19097 (time 1773941510.35 total 42.89 last 452.33) - open/close 20000 (time 1773941512.37 total 44.91 last 446.52) - open/close 23976 (time 1773941522.39 total 54.92 last 397.15) - open/close 28181 (time 1773941532.39 total 64.92 last 420.43) - open/close 30000 (time 1773941536.25 total 68.79 last 470.33) - open/close 34314 (time 1773941546.26 total 78.79 last 431.36) - open/close 38753 (time 1773941556.26 total 88.79 last 443.88) - open/close 40000 (time 1773941559.17 total 91.71 last 427.31) - open/close 44395 (time 1773941569.17 total 101.71 last 439.49) - open/close 48354 (time 1773941579.18 total 111.71 last 395.80) - open/close 50000 (time 1773941585.16 total 117.69 last 275.19) - open/close 53026 (time 1773941595.16 total 127.69 last 302.56) - open/close 56427 (time 1773941605.16 total 137.70 last 340.03) - open/close 60000 (time 1773941614.38 total 146.92 last 387.52) - open/close 63024 (time 1773941624.38 total 156.92 last 302.35) total: 64768 open/close in 162.74 seconds: 397.99 ops/second - /unlink 5246 (time 1773941641.87 total 10.00 last 524.54) - /unlink 9586 (time 1773941651.87 total 20.00 last 433.95) - /unlink 14961 (time 1773941661.87 total 30.00 last 537.47) - /unlink 20000 (time 1773941671.11 total 39.24 last 545.59) - /unlink 24742 (time 1773941681.11 total 49.24 last 474.14) - /unlink 30000 (time 1773941690.39 total 58.52 last 566.46) - /unlink 35006 (time 1773941700.39 total 68.52 last 500.57) - /unlink 40000 (time 1773941709.63 total 77.77 last 540.30) - /unlink 45473 (time 1773941719.64 total 87.77 last 547.26) - /unlink 50000 (time 1773941728.20 total 96.33 last 528.89) - /unlink 55879 (time 1773941738.20 total 106.33 last 587.83) - /unlink 60000 (time 1773941746.66 total 114.80 last 486.65) - /unlink 64515 (time 1773941756.67 total 124.80 last 451.39) total: 64768 /unlink in 125.48 seconds: 516.16 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 3234 (time 1773941791.11 total 10.00 last 323.39) - open/close 6402 (time 1773941801.11 total 20.00 last 316.75) - open/close 9608 (time 1773941811.11 total 30.00 last 320.59) - open/close 12540 (time 1773941821.11 total 40.00 last 293.19) - open/close 15323 (time 1773941831.11 total 50.00 last 278.27) - open/close 17998 (time 1773941841.11 total 60.00 last 267.46) - open/close 20000 (time 1773941847.73 total 66.62 last 302.67) - open/close 22801 (time 1773941857.73 total 76.62 last 280.01) - open/close 25244 (time 1773941867.73 total 86.62 last 244.25) - open/close 27541 (time 1773941877.74 total 96.63 last 229.67) - open/close 29901 (time 1773941887.74 total 106.63 last 235.98) - open/close 32274 (time 1773941897.74 total 116.63 last 237.26) - open/close 35109 (time 1773941907.74 total 126.63 last 283.50) - open/close 37965 (time 1773941917.74 total 136.63 last 285.55) - open/close 40000 (time 1773941924.85 total 143.74 last 286.12) - open/close 42523 (time 1773941934.85 total 153.74 last 252.28) - open/close 45000 (time 1773941944.85 total 163.74 last 247.65) - open/close 47786 (time 1773941954.86 total 173.75 last 278.58) - open/close 50000 (time 1773941962.18 total 181.07 last 302.13) - open/close 53028 (time 1773941972.18 total 191.07 last 302.77) - open/close 56238 (time 1773941982.19 total 201.08 last 320.96) - open/close 58731 (time 1773941992.19 total 211.08 last 249.25) - open/close 60000 (time 1773941996.94 total 215.84 last 266.74) - open/close 62554 (time 1773942006.95 total 225.84 last 255.32) total: 64768 open/close in 234.39 seconds: 276.33 ops/second - /unlink 6222 (time 1773942027.14 total 10.00 last 622.17) - /unlink 10000 (time 1773942034.34 total 17.19 last 525.34) - /unlink 14920 (time 1773942044.34 total 27.19 last 491.97) - /unlink 20000 (time 1773942054.22 total 37.08 last 513.89) - /unlink 25443 (time 1773942064.22 total 47.08 last 544.23) - /unlink 30000 (time 1773942073.23 total 56.08 last 506.05) - /unlink 35277 (time 1773942083.23 total 66.08 last 527.69) - /unlink 40000 (time 1773942092.73 total 75.59 last 496.91) - /unlink 45097 (time 1773942102.73 total 85.59 last 509.64) - /unlink 49875 (time 1773942112.74 total 95.60 last 477.39) - /unlink 54179 (time 1773942122.74 total 105.60 last 430.32) - /unlink 58742 (time 1773942132.74 total 115.60 last 456.27) - /unlink 60000 (time 1773942135.76 total 118.62 last 417.08) total: 64768 /unlink in 127.97 seconds: 506.12 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2358 (time 1773942173.13 total 10.01 last 235.67) - open/close 5314 (time 1773942183.13 total 20.01 last 295.59) - open/close 7963 (time 1773942193.13 total 30.01 last 264.83) - open/close 10000 (time 1773942201.58 total 38.46 last 241.10) - open/close 12482 (time 1773942211.58 total 48.46 last 248.11) - open/close 14747 (time 1773942221.58 total 58.46 last 226.50) - open/close 17115 (time 1773942231.58 total 68.46 last 236.78) - open/close 19394 (time 1773942241.58 total 78.46 last 227.87) - open/close 20000 (time 1773942243.59 total 80.47 last 301.85) - open/close 22808 (time 1773942253.59 total 90.47 last 280.76) - open/close 25853 (time 1773942263.59 total 100.47 last 304.45) - open/close 29650 (time 1773942273.59 total 110.48 last 379.66) - open/close 32146 (time 1773942283.60 total 120.48 last 249.59) - open/close 34877 (time 1773942293.60 total 130.48 last 273.07) - open/close 38826 (time 1773942303.60 total 140.48 last 394.88) - open/close 40000 (time 1773942308.30 total 145.18 last 249.67) - open/close 42323 (time 1773942318.30 total 155.18 last 232.29) - open/close 45746 (time 1773942328.30 total 165.18 last 342.28) - open/close 50000 (time 1773942338.14 total 175.02 last 432.22) - open/close 54351 (time 1773942348.14 total 185.02 last 435.06) - open/close 58112 (time 1773942358.15 total 195.03 last 375.96) - open/close 60000 (time 1773942365.82 total 202.70 last 245.98) - open/close 63702 (time 1773942375.82 total 212.70 last 370.16) total: 64768 open/close in 215.52 seconds: 300.51 ops/second - /unlink 6802 (time 1773942390.54 total 10.00 last 680.17) - /unlink 10000 (time 1773942395.95 total 15.41 last 590.86) - /unlink 17341 (time 1773942405.95 total 25.41 last 734.06) - /unlink 20000 (time 1773942409.74 total 29.21 last 701.12) - /unlink 26797 (time 1773942419.74 total 39.21 last 679.67) - /unlink 30000 (time 1773942424.30 total 43.76 last 703.19) - /unlink 35470 (time 1773942434.30 total 53.76 last 546.97) - /unlink 40000 (time 1773942441.73 total 61.20 last 609.38) - /unlink 46179 (time 1773942451.73 total 71.20 last 617.79) - /unlink 50000 (time 1773942459.59 total 79.05 last 486.65) - /unlink 55177 (time 1773942469.59 total 89.05 last 517.65) - /unlink 60000 (time 1773942479.18 total 98.65 last 502.65) total: 64768 /unlink in 108.24 seconds: 598.40 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 2685 (time 1773942516.06 total 10.00 last 268.45) - open/close 5492 (time 1773942526.06 total 20.00 last 280.67) - open/close 8135 (time 1773942536.07 total 30.00 last 264.29) - open/close 10000 (time 1773942542.98 total 36.92 last 269.54) - open/close 12617 (time 1773942552.99 total 46.93 last 261.62) - open/close 14949 (time 1773942562.99 total 56.93 last 233.14) - open/close 17779 (time 1773942573.01 total 66.95 last 282.36) - open/close 20000 (time 1773942582.30 total 76.24 last 239.11) - open/close 22638 (time 1773942592.30 total 86.24 last 263.72) - open/close 25417 (time 1773942602.31 total 96.25 last 277.79) - open/close 27793 (time 1773942612.31 total 106.25 last 237.52) - open/close 30000 (time 1773942621.53 total 115.47 last 239.30) - open/close 32399 (time 1773942631.54 total 125.47 last 239.87) - open/close 35100 (time 1773942641.54 total 135.47 last 270.09) - open/close 38097 (time 1773942651.54 total 145.48 last 299.65) - open/close 40000 (time 1773942658.78 total 152.72 last 262.75) - open/close 42757 (time 1773942668.78 total 162.72 last 275.61) - open/close 45279 (time 1773942678.79 total 172.73 last 252.10) - open/close 47839 (time 1773942688.79 total 182.73 last 255.90) - open/close 50000 (time 1773942696.89 total 190.82 last 267.01) - open/close 52825 (time 1773942706.89 total 200.83 last 282.40) - open/close 55351 (time 1773942716.90 total 210.83 last 252.38) - open/close 57723 (time 1773942726.98 total 220.91 last 235.35) - open/close 60000 (time 1773942736.27 total 230.20 last 245.09) - open/close 62347 (time 1773942746.27 total 240.21 last 234.64) - open/close 64503 (time 1773942756.27 total 250.21 last 215.54) total: 64768 open/close in 251.53 seconds: 257.50 ops/second - /unlink 4453 (time 1773942769.67 total 10.00 last 445.21) - /unlink 8842 (time 1773942779.67 total 20.00 last 438.87) - /unlink 10000 (time 1773942781.95 total 22.27 last 509.62) - /unlink 14512 (time 1773942791.95 total 32.28 last 451.14) - /unlink 18525 (time 1773942801.95 total 42.28 last 401.23) - /unlink 20000 (time 1773942805.31 total 45.64 last 439.14) - /unlink 24469 (time 1773942815.31 total 55.64 last 446.88) - /unlink 29417 (time 1773942825.31 total 65.64 last 494.75) - /unlink 33980 (time 1773942835.31 total 75.64 last 456.28) - /unlink 38167 (time 1773942845.31 total 85.64 last 418.70) - /unlink 40000 (time 1773942850.16 total 90.49 last 377.86) - /unlink 44368 (time 1773942860.16 total 100.49 last 436.78) - /unlink 48541 (time 1773942870.16 total 110.49 last 417.29) - /unlink 50000 (time 1773942873.04 total 113.37 last 506.80) - /unlink 54587 (time 1773942883.04 total 123.37 last 458.62) - /unlink 59196 (time 1773942893.04 total 133.37 last 460.88) - /unlink 64496 (time 1773942903.04 total 143.37 last 529.91) total: 64768 /unlink in 143.87 seconds: 450.17 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 1936 (time 1773942936.50 total 10.00 last 193.55) - open/close 3970 (time 1773942946.50 total 20.00 last 203.35) - open/close 4951 (time 1773942956.51 total 30.01 last 98.05) - open/close 7586 (time 1773942966.51 total 40.01 last 263.50) - open/close 10000 (time 1773942972.57 total 46.07 last 398.03) - open/close 14524 (time 1773942982.57 total 56.08 last 452.34) - open/close 18893 (time 1773942992.57 total 66.08 last 436.85) - open/close 20000 (time 1773942996.08 total 69.58 last 315.63) - open/close 23082 (time 1773943006.08 total 79.59 last 308.14) - open/close 27191 (time 1773943016.08 total 89.59 last 410.88) - open/close 30000 (time 1773943022.64 total 96.14 last 428.54) - open/close 34839 (time 1773943032.64 total 106.14 last 483.84) - open/close 39460 (time 1773943042.64 total 116.14 last 462.06) - open/close 44086 (time 1773943052.64 total 126.15 last 462.52) - open/close 47533 (time 1773943062.64 total 136.15 last 344.68) - open/close 50000 (time 1773943069.42 total 142.93 last 363.91) - open/close 53618 (time 1773943079.42 total 152.93 last 361.73) - open/close 57845 (time 1773943089.43 total 162.93 last 422.68) - open/close 60000 (time 1773943095.54 total 169.04 last 352.45) - open/close 63717 (time 1773943105.54 total 179.04 last 371.68) total: 64768 open/close in 182.16 seconds: 355.56 ops/second - /unlink 6369 (time 1773943120.03 total 10.00 last 636.90) - /unlink 10000 (time 1773943125.50 total 15.47 last 663.71) - /unlink 17120 (time 1773943135.50 total 25.47 last 711.92) - /unlink 20000 (time 1773943139.44 total 29.41 last 731.16) - /unlink 26887 (time 1773943149.44 total 39.41 last 688.69) - /unlink 30000 (time 1773943154.06 total 44.03 last 673.44) - /unlink 36595 (time 1773943164.06 total 54.03 last 659.42) - /unlink 40000 (time 1773943171.46 total 61.44 last 459.86) - /unlink 44558 (time 1773943181.47 total 71.44 last 455.76) - /unlink 49381 (time 1773943191.47 total 81.44 last 482.26) - /unlink 54851 (time 1773943201.47 total 91.44 last 546.87) - /unlink 59034 (time 1773943211.47 total 101.44 last 418.25) - /unlink 60000 (time 1773943213.63 total 103.60 last 447.44) - /unlink 64431 (time 1773943223.63 total 113.60 last 443.03) total: 64768 /unlink in 114.37 seconds: 566.28 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg260-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg260-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg260-server stop mds service on oleg260-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg260-server unloading modules via unload_modules_local on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing unload_modules_local oleg260-server: modules unloaded. pdsh@oleg260-client: oleg260-server: ssh exited with exit code 2 pdsh@oleg260-client: oleg260-client: ssh exited with exit code 2 pdsh@oleg260-client: oleg260-server: ssh exited with exit code 2 pdsh@oleg260-client: oleg260-client: ssh exited with exit code 2 PASS 106 (1829s) Stopping clients: oleg260-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg260-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg260-client: oleg260-server: ssh exited with exit code 2 oleg260-server: oleg260-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg260-server' oleg260-server: oleg260-server.virtnet: executing load_modules_local oleg260-server: Loading modules from /home/green/git/lustre-release/lustre oleg260-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1040: echo: write error: Device or resource busy oleg260-server: detected 4 online CPUs by sysfs oleg260-server: MODOPTS_LIBCFS= oleg260-server: Force libcfs to create 2 CPU partitions oleg260-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg260-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg260-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1944 sec ========== 14:01:34 (1773943294) === conf-sanity: start cleanup 14:01:34 (1773943294) === === conf-sanity: finish cleanup 14:01:36 (1773943296) ===