-----============= acceptance-small: conf-sanity ============----- Tue Jun 10 21:31:11 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg141-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg141-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg141-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg141-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg141-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg141-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg141-server oleg141-server: oleg141-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg141-server' oleg141-server: oleg141-server.virtnet: executing load_modules_local oleg141-server: Loading modules from /home/green/git/lustre-release/lustre oleg141-server: detected 4 online CPUs by sysfs oleg141-server: MODOPTS_LIBCFS= oleg141-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg141-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg141-server: oleg141-server.virtnet: executing set_default_debug -1 all pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg141-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg141-server: oleg141-server.virtnet: executing set_default_debug -1 all pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg141-server: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg141-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg141-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg141-server stop mds service on oleg141-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg141-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 21:32:09 (1749605529) Stopping clients: oleg141-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg141-client.virtnet /mnt/lustre2 (opts:-f) oleg141-server: oleg141-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg141-server' oleg141-server: oleg141-server.virtnet: executing load_modules_local oleg141-server: Loading modules from /home/green/git/lustre-release/lustre oleg141-server: detected 4 online CPUs by sysfs oleg141-server: MODOPTS_LIBCFS= oleg141-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg141-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg141-server: oleg141-server.virtnet: executing set_default_debug -1 all pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg141-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg141-server: oleg141-server.virtnet: executing set_default_debug -1 all pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg141-client.virtnet: -o user_xattr,flock 192.168.201.141@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 3976 (time 1749605566.70 total 10.00 last 397.49) - open/close 7776 (time 1749605576.70 total 20.01 last 379.89) - open/close 10000 (time 1749605582.16 total 25.46 last 407.40) - open/close 14162 (time 1749605592.16 total 35.47 last 416.17) - open/close 18329 (time 1749605602.16 total 45.47 last 416.67) - open/close 20000 (time 1749605606.93 total 50.23 last 350.85) - open/close 23846 (time 1749605616.93 total 60.23 last 384.46) - open/close 28744 (time 1749605626.93 total 70.23 last 489.72) - open/close 30000 (time 1749605629.73 total 73.03 last 448.51) - open/close 33756 (time 1749605639.73 total 83.04 last 375.58) - open/close 39027 (time 1749605649.73 total 93.04 last 527.07) - open/close 43585 (time 1749605659.73 total 103.04 last 455.71) - open/close 48387 (time 1749605669.74 total 113.04 last 480.17) - open/close 50000 (time 1749605673.20 total 116.50 last 465.96) - open/close 54917 (time 1749605683.20 total 126.50 last 491.62) - open/close 59103 (time 1749605693.20 total 136.50 last 418.58) - open/close 63620 (time 1749605703.20 total 146.50 last 451.62) total: 64768 open/close in 150.96 seconds: 429.05 ops/second - /unlink 5099 (time 1749605718.89 total 10.00 last 509.83) - /unlink 10000 (time 1749605728.14 total 19.24 last 530.30) - /unlink 15919 (time 1749605738.14 total 29.24 last 591.88) - /unlink 20000 (time 1749605746.86 total 37.97 last 467.66) - /unlink 26359 (time 1749605756.86 total 47.97 last 635.77) - /unlink 30000 (time 1749605762.50 total 53.61 last 646.10) - /unlink 37373 (time 1749605772.50 total 63.61 last 737.25) - /unlink 40000 (time 1749605776.40 total 67.51 last 673.48) - /unlink 48070 (time 1749605786.40 total 77.51 last 806.92) - /unlink 50000 (time 1749605788.60 total 79.71 last 878.92) - /unlink 59197 (time 1749605798.60 total 89.71 last 919.63) total: 64768 /unlink in 95.77 seconds: 676.31 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5547 (time 1749605834.30 total 10.00 last 554.68) - open/close 10000 (time 1749605843.42 total 19.11 last 488.58) - open/close 14651 (time 1749605853.42 total 29.12 last 465.07) - open/close 19255 (time 1749605863.42 total 39.12 last 460.32) - open/close 24199 (time 1749605873.42 total 49.12 last 494.37) - open/close 28946 (time 1749605883.42 total 59.12 last 474.67) - open/close 30000 (time 1749605885.50 total 61.20 last 505.56) - open/close 34522 (time 1749605895.51 total 71.20 last 452.12) - open/close 39183 (time 1749605905.51 total 81.20 last 466.09) - open/close 43613 (time 1749605915.51 total 91.20 last 443.00) - open/close 47797 (time 1749605925.51 total 101.20 last 418.40) - open/close 50000 (time 1749605930.93 total 106.63 last 406.24) - open/close 53950 (time 1749605940.93 total 116.63 last 394.99) - open/close 57761 (time 1749605950.93 total 126.63 last 381.08) - open/close 60000 (time 1749605956.71 total 132.41 last 387.17) - open/close 63314 (time 1749605966.71 total 142.41 last 331.34) total: 64768 open/close in 146.20 seconds: 443.01 ops/second - /unlink 6857 (time 1749605981.54 total 10.00 last 685.65) - /unlink 10000 (time 1749605986.16 total 14.61 last 681.66) - /unlink 17380 (time 1749605996.16 total 24.61 last 737.99) - /unlink 20000 (time 1749606000.78 total 29.23 last 566.88) - /unlink 25566 (time 1749606010.78 total 39.23 last 556.51) - /unlink 30000 (time 1749606018.15 total 46.61 last 601.53) - /unlink 35576 (time 1749606028.15 total 56.61 last 557.59) - /unlink 40000 (time 1749606037.34 total 65.80 last 481.45) - /unlink 44843 (time 1749606047.34 total 75.80 last 484.21) - /unlink 50000 (time 1749606056.93 total 85.39 last 537.62) - /unlink 55182 (time 1749606066.93 total 95.39 last 518.14) - /unlink 60000 (time 1749606073.57 total 102.03 last 725.65) total: 64768 /unlink in 110.16 seconds: 587.94 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 3220 (time 1749606109.60 total 10.00 last 321.96) - open/close 6770 (time 1749606119.60 total 20.00 last 354.96) - open/close 10000 (time 1749606127.33 total 27.73 last 417.90) - open/close 13334 (time 1749606137.33 total 37.73 last 333.37) - open/close 17718 (time 1749606147.33 total 47.73 last 438.35) - open/close 20000 (time 1749606156.10 total 56.50 last 260.16) - open/close 24250 (time 1749606166.10 total 66.51 last 424.96) - open/close 28015 (time 1749606176.11 total 76.51 last 376.41) - open/close 30000 (time 1749606180.21 total 80.61 last 483.76) - open/close 33858 (time 1749606190.21 total 90.61 last 385.73) - open/close 37309 (time 1749606200.21 total 100.62 last 344.99) - open/close 40000 (time 1749606207.75 total 108.15 last 357.23) - open/close 44146 (time 1749606217.75 total 118.15 last 414.60) - open/close 48620 (time 1749606227.75 total 128.15 last 447.29) - open/close 50000 (time 1749606231.80 total 132.21 last 340.36) - open/close 52779 (time 1749606241.81 total 142.21 last 277.85) - open/close 55520 (time 1749606251.81 total 152.21 last 274.07) - open/close 58359 (time 1749606261.81 total 162.21 last 283.88) - open/close 60000 (time 1749606267.81 total 168.21 last 273.58) - open/close 62809 (time 1749606277.81 total 178.21 last 280.90) total: 64768 open/close in 185.37 seconds: 349.40 ops/second - /unlink 4712 (time 1749606295.78 total 10.00 last 471.11) - /unlink 9353 (time 1749606305.78 total 20.00 last 464.00) - /unlink 14031 (time 1749606315.78 total 30.00 last 467.75) - /unlink 19046 (time 1749606325.78 total 40.01 last 501.45) - /unlink 20000 (time 1749606327.93 total 42.15 last 445.74) - /unlink 26301 (time 1749606337.93 total 52.15 last 629.98) - /unlink 30000 (time 1749606343.88 total 58.10 last 621.33) - /unlink 34971 (time 1749606353.88 total 68.10 last 497.05) - /unlink 40000 (time 1749606362.51 total 76.73 last 583.02) - /unlink 45198 (time 1749606372.51 total 86.73 last 519.73) - /unlink 50000 (time 1749606379.81 total 94.03 last 657.46) - /unlink 56044 (time 1749606389.81 total 104.03 last 604.35) - /unlink 60000 (time 1749606397.94 total 112.16 last 486.75) total: 64768 /unlink in 119.48 seconds: 542.07 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 3771 (time 1749606435.13 total 10.00 last 377.09) - open/close 6833 (time 1749606445.13 total 20.00 last 306.11) - open/close 9713 (time 1749606455.13 total 30.00 last 287.97) - open/close 12394 (time 1749606465.14 total 40.01 last 268.03) - open/close 14956 (time 1749606475.14 total 50.01 last 256.19) - open/close 17516 (time 1749606485.14 total 60.01 last 255.92) - open/close 20000 (time 1749606494.45 total 69.32 last 266.84) - open/close 22875 (time 1749606504.45 total 79.32 last 287.40) - open/close 26171 (time 1749606514.45 total 89.32 last 329.57) - open/close 29212 (time 1749606524.46 total 99.33 last 304.05) - open/close 30000 (time 1749606527.51 total 102.38 last 257.75) - open/close 33020 (time 1749606537.51 total 112.38 last 301.97) - open/close 35927 (time 1749606547.52 total 122.39 last 290.58) - open/close 38500 (time 1749606557.52 total 132.39 last 257.25) - open/close 40000 (time 1749606563.24 total 138.11 last 262.35) - open/close 42934 (time 1749606573.24 total 148.11 last 293.34) - open/close 45799 (time 1749606583.24 total 158.11 last 286.41) - open/close 48633 (time 1749606593.25 total 168.11 last 283.34) - open/close 50000 (time 1749606598.51 total 173.38 last 259.53) - open/close 52913 (time 1749606608.52 total 183.39 last 291.20) - open/close 55413 (time 1749606618.52 total 193.39 last 249.93) - open/close 58282 (time 1749606628.52 total 203.39 last 286.80) - open/close 60000 (time 1749606634.48 total 209.35 last 288.24) - open/close 63061 (time 1749606644.48 total 219.35 last 306.06) total: 64768 open/close in 224.81 seconds: 288.10 ops/second - /unlink 5276 (time 1749606661.17 total 10.00 last 527.59) - /unlink 9901 (time 1749606671.18 total 20.00 last 462.43) - /unlink 14498 (time 1749606681.18 total 30.00 last 459.65) - /unlink 19032 (time 1749606691.18 total 40.00 last 453.37) - /unlink 20000 (time 1749606693.39 total 42.21 last 438.32) - /unlink 24679 (time 1749606703.39 total 52.21 last 467.81) - /unlink 29262 (time 1749606713.39 total 62.21 last 458.26) - /unlink 33899 (time 1749606723.39 total 72.22 last 463.63) - /unlink 38463 (time 1749606733.39 total 82.22 last 456.36) - /unlink 40000 (time 1749606736.67 total 85.50 last 468.57) - /unlink 44551 (time 1749606746.67 total 95.50 last 455.00) - /unlink 48990 (time 1749606756.68 total 105.50 last 443.86) - /unlink 50000 (time 1749606759.04 total 107.87 last 426.81) - /unlink 54561 (time 1749606769.04 total 117.87 last 456.04) - /unlink 59277 (time 1749606779.04 total 127.87 last 471.51) - /unlink 64092 (time 1749606789.04 total 137.87 last 481.49) total: 64768 /unlink in 139.07 seconds: 465.73 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 3371 (time 1749606815.69 total 10.00 last 336.98) - open/close 8040 (time 1749606825.69 total 20.00 last 466.84) - open/close 10000 (time 1749606829.74 total 24.06 last 483.46) - open/close 15005 (time 1749606839.74 total 34.06 last 500.41) - open/close 20000 (time 1749606849.30 total 43.62 last 522.79) - open/close 25574 (time 1749606859.30 total 53.62 last 557.33) - open/close 30000 (time 1749606867.33 total 61.65 last 551.12) - open/close 35455 (time 1749606877.33 total 71.65 last 545.41) - open/close 40000 (time 1749606885.40 total 79.72 last 563.24) - open/close 45376 (time 1749606895.40 total 89.72 last 537.58) - open/close 50000 (time 1749606903.94 total 98.26 last 541.37) - open/close 54346 (time 1749606913.94 total 108.26 last 434.55) - open/close 59921 (time 1749606923.94 total 118.26 last 557.49) total: 64768 open/close in 126.93 seconds: 510.28 ops/second - /unlink 8012 (time 1749606943.35 total 10.00 last 801.19) - /unlink 10000 (time 1749606946.12 total 12.77 last 718.22) - /unlink 18551 (time 1749606956.12 total 22.77 last 855.08) - /unlink 27813 (time 1749606966.12 total 32.77 last 926.18) - /unlink 30000 (time 1749606969.08 total 35.73 last 738.92) - /unlink 37928 (time 1749606979.08 total 45.73 last 792.74) - /unlink 40000 (time 1749606981.39 total 48.04 last 897.80) - /unlink 48065 (time 1749606991.39 total 58.04 last 806.42) - /unlink 50000 (time 1749606993.39 total 60.04 last 964.63) - /unlink 58695 (time 1749607003.40 total 70.04 last 869.41) total: 64768 /unlink in 76.30 seconds: 848.86 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg141-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg141-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg141-server stop mds service on oleg141-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg141-server unloading modules via unload_modules_local on: 'oleg141-server' oleg141-server: oleg141-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg141-client: oleg141-client: ssh exited with exit code 2 pdsh@oleg141-client: oleg141-server: ssh exited with exit code 2 pdsh@oleg141-client: oleg141-client: ssh exited with exit code 2 pdsh@oleg141-client: oleg141-server: ssh exited with exit code 2 PASS 106 (1521s) Stopping clients: oleg141-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg141-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg141-client: oleg141-server: ssh exited with exit code 2 oleg141-server: oleg141-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg141-server' oleg141-server: oleg141-server.virtnet: executing load_modules_local oleg141-server: Loading modules from /home/green/git/lustre-release/lustre oleg141-server: detected 4 online CPUs by sysfs oleg141-server: MODOPTS_LIBCFS= oleg141-server: Force libcfs to create 2 CPU partitions oleg141-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg141-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1596 sec ========== 21:57:47 (1749607067) === conf-sanity: start cleanup 21:57:47 (1749607067) === === conf-sanity: finish cleanup 21:57:48 (1749607068) ===