-----============= acceptance-small: conf-sanity ============----- Fri May 9 07:48:09 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg405-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg405-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg405-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg405-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg405-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg405-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg405-server oleg405-server: oleg405-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: detected 4 online CPUs by sysfs oleg405-server: MODOPTS_LIBCFS= oleg405-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg405-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg405-server: oleg405-server.virtnet: executing set_default_debug -1 all pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg405-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg405-server: oleg405-server.virtnet: executing set_default_debug -1 all pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg405-server: oleg405-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg405-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg405-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg405-server stop mds service on oleg405-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg405-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 07:49:06 (1746791346) Stopping clients: oleg405-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg405-client.virtnet /mnt/lustre2 (opts:-f) oleg405-server: oleg405-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: detected 4 online CPUs by sysfs oleg405-server: MODOPTS_LIBCFS= oleg405-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg405-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg405-server: oleg405-server.virtnet: executing set_default_debug -1 all pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg405-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg405-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg405-server: oleg405-server.virtnet: executing set_default_debug -1 all pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg405-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg405-client.virtnet: -o user_xattr,flock 192.168.204.105@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86890 331 86559 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32587 331 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 5003 (time 1746791380.22 total 10.00 last 500.18) - open/close 10000 (time 1746791389.88 total 19.67 last 517.10) - open/close 15208 (time 1746791399.88 total 29.67 last 520.75) - open/close 20000 (time 1746791409.04 total 38.83 last 523.22) - open/close 25295 (time 1746791419.04 total 48.83 last 529.42) - open/close 30000 (time 1746791427.74 total 57.53 last 540.70) - open/close 35386 (time 1746791437.75 total 67.53 last 538.54) - open/close 40000 (time 1746791446.18 total 75.96 last 547.35) - open/close 45016 (time 1746791456.18 total 85.96 last 501.54) - open/close 49832 (time 1746791466.18 total 95.96 last 481.51) - open/close 54581 (time 1746791476.18 total 105.96 last 474.84) - open/close 59764 (time 1746791486.18 total 115.96 last 518.26) total: 64768 open/close in 125.32 seconds: 516.83 ops/second - /unlink 8988 (time 1746791506.37 total 10.00 last 898.78) - /unlink 18311 (time 1746791516.37 total 20.00 last 932.21) - /unlink 26020 (time 1746791526.37 total 30.00 last 770.84) - /unlink 30000 (time 1746791530.70 total 34.33 last 919.05) - /unlink 38921 (time 1746791540.70 total 44.33 last 892.03) - /unlink 47257 (time 1746791550.70 total 54.34 last 833.45) - /unlink 50000 (time 1746791554.07 total 57.71 last 813.38) - /unlink 58184 (time 1746791564.07 total 67.71 last 818.39) total: 64768 /unlink in 74.62 seconds: 868.01 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5426 (time 1746791603.78 total 10.00 last 542.58) - open/close 10000 (time 1746791612.67 total 18.88 last 514.82) - open/close 15171 (time 1746791622.67 total 28.89 last 517.09) - open/close 20000 (time 1746791631.98 total 38.20 last 518.47) - open/close 25158 (time 1746791641.98 total 48.20 last 515.77) - open/close 30000 (time 1746791651.80 total 58.02 last 493.07) - open/close 35048 (time 1746791661.80 total 68.02 last 504.79) - open/close 40000 (time 1746791671.52 total 77.74 last 509.66) - open/close 45158 (time 1746791681.52 total 87.74 last 515.72) - open/close 50000 (time 1746791691.07 total 97.29 last 506.96) - open/close 55198 (time 1746791701.07 total 107.29 last 519.70) - open/close 60000 (time 1746791710.45 total 116.67 last 511.94) total: 64768 open/close in 125.76 seconds: 515.00 ops/second - /unlink 9037 (time 1746791730.38 total 10.00 last 903.68) - /unlink 17971 (time 1746791740.38 total 20.00 last 893.34) - /unlink 20000 (time 1746791742.58 total 22.20 last 922.25) - /unlink 29050 (time 1746791752.58 total 32.20 last 904.95) - /unlink 38196 (time 1746791762.59 total 42.20 last 914.46) - /unlink 40000 (time 1746791764.59 total 44.20 last 901.55) - /unlink 48965 (time 1746791774.59 total 54.21 last 896.38) - /unlink 57400 (time 1746791784.59 total 64.21 last 843.43) - /unlink 60000 (time 1746791787.30 total 66.92 last 959.11) total: 64768 /unlink in 72.84 seconds: 889.20 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5288 (time 1746791829.89 total 10.00 last 528.75) - open/close 10000 (time 1746791839.39 total 19.50 last 496.17) - open/close 15259 (time 1746791849.39 total 29.50 last 525.81) - open/close 19870 (time 1746791859.39 total 39.50 last 461.08) - open/close 24972 (time 1746791869.39 total 49.50 last 510.12) - open/close 30000 (time 1746791879.02 total 59.13 last 521.94) - open/close 34729 (time 1746791889.03 total 69.14 last 472.74) - open/close 38636 (time 1746791899.03 total 79.14 last 390.61) - open/close 40000 (time 1746791903.19 total 83.30 last 327.97) - open/close 43485 (time 1746791913.19 total 93.30 last 348.45) - open/close 48009 (time 1746791923.19 total 103.30 last 452.31) - open/close 50000 (time 1746791927.77 total 107.88 last 435.18) - open/close 54878 (time 1746791937.77 total 117.88 last 487.72) - open/close 59704 (time 1746791947.77 total 127.88 last 482.56) - open/close 64533 (time 1746791957.77 total 137.88 last 482.81) total: 64768 open/close in 138.39 seconds: 468.01 ops/second - /unlink 8432 (time 1746791969.11 total 10.00 last 843.14) - /unlink 17514 (time 1746791979.11 total 20.00 last 908.15) - /unlink 20000 (time 1746791981.99 total 22.88 last 864.97) - /unlink 28756 (time 1746791991.99 total 32.88 last 875.45) - /unlink 37990 (time 1746792001.99 total 42.88 last 923.32) - /unlink 40000 (time 1746792004.22 total 45.11 last 902.37) - /unlink 49120 (time 1746792014.22 total 55.11 last 911.97) - /unlink 57480 (time 1746792024.22 total 65.11 last 835.92) - /unlink 60000 (time 1746792027.13 total 68.02 last 866.37) total: 64768 /unlink in 72.98 seconds: 887.48 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5021 (time 1746792070.23 total 10.00 last 502.08) - open/close 9449 (time 1746792080.23 total 20.00 last 442.77) - open/close 13966 (time 1746792090.23 total 30.00 last 451.63) - open/close 18435 (time 1746792100.23 total 40.00 last 446.90) - open/close 20000 (time 1746792103.22 total 42.99 last 523.97) - open/close 25057 (time 1746792113.22 total 52.99 last 505.62) - open/close 29939 (time 1746792123.22 total 62.99 last 488.18) - open/close 35118 (time 1746792133.22 total 72.99 last 517.86) - open/close 40000 (time 1746792142.83 total 82.60 last 508.27) - open/close 45115 (time 1746792152.83 total 92.60 last 511.42) - open/close 49809 (time 1746792162.83 total 102.60 last 469.37) - open/close 53853 (time 1746792172.83 total 112.60 last 404.38) - open/close 58958 (time 1746792182.83 total 122.60 last 510.50) - open/close 63382 (time 1746792192.83 total 132.60 last 442.34) total: 64768 open/close in 135.48 seconds: 478.07 ops/second - /unlink 8520 (time 1746792206.55 total 10.00 last 851.96) - /unlink 17281 (time 1746792216.55 total 20.00 last 876.06) - /unlink 20000 (time 1746792220.46 total 23.91 last 695.62) - /unlink 26355 (time 1746792230.46 total 33.91 last 635.38) - /unlink 30000 (time 1746792235.15 total 38.59 last 778.35) - /unlink 37852 (time 1746792245.15 total 48.60 last 785.11) - /unlink 40000 (time 1746792248.18 total 51.63 last 708.55) - /unlink 47091 (time 1746792258.18 total 61.63 last 709.08) - /unlink 50000 (time 1746792263.15 total 66.60 last 585.19) - /unlink 56479 (time 1746792273.15 total 76.60 last 647.89) - /unlink 60000 (time 1746792277.87 total 81.31 last 746.63) total: 64768 /unlink in 88.57 seconds: 731.30 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5038 (time 1746792315.51 total 10.00 last 503.69) - open/close 10000 (time 1746792324.93 total 19.43 last 526.43) - open/close 15135 (time 1746792334.93 total 29.43 last 513.49) - open/close 19350 (time 1746792344.94 total 39.43 last 421.45) - open/close 23716 (time 1746792354.94 total 49.43 last 436.59) - open/close 27936 (time 1746792364.94 total 59.43 last 421.96) - open/close 30000 (time 1746792370.01 total 64.51 last 406.65) - open/close 33585 (time 1746792380.01 total 74.51 last 358.48) - open/close 38069 (time 1746792390.01 total 84.51 last 448.39) - open/close 40000 (time 1746792395.21 total 89.70 last 371.52) - open/close 44094 (time 1746792405.21 total 99.71 last 409.33) - open/close 47122 (time 1746792415.22 total 109.71 last 302.68) - open/close 49774 (time 1746792425.22 total 119.71 last 265.12) - open/close 52963 (time 1746792435.22 total 129.71 last 318.86) - open/close 56051 (time 1746792445.22 total 139.72 last 308.74) - open/close 58781 (time 1746792455.22 total 149.72 last 272.96) - open/close 60000 (time 1746792459.21 total 153.70 last 306.11) - open/close 63634 (time 1746792469.21 total 163.70 last 363.33) total: 64768 open/close in 167.05 seconds: 387.71 ops/second - /unlink 6246 (time 1746792483.66 total 10.00 last 624.52) - /unlink 10000 (time 1746792489.15 total 15.50 last 683.21) - /unlink 16013 (time 1746792499.16 total 25.50 last 601.25) - /unlink 20000 (time 1746792504.37 total 30.71 last 765.11) - /unlink 27197 (time 1746792514.37 total 40.71 last 719.66) - /unlink 30000 (time 1746792517.92 total 44.26 last 788.76) - /unlink 37687 (time 1746792527.92 total 54.26 last 768.67) - /unlink 40000 (time 1746792530.73 total 57.07 last 822.87) - /unlink 48551 (time 1746792540.73 total 67.07 last 854.96) - /unlink 50000 (time 1746792543.32 total 69.66 last 561.18) - /unlink 56251 (time 1746792553.32 total 79.66 last 625.09) - /unlink 60000 (time 1746792558.37 total 84.71 last 742.03) total: 64768 /unlink in 90.26 seconds: 717.57 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg405-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg405-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg405-server stop mds service on oleg405-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg405-server unloading modules via unload_modules_local on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg405-client: oleg405-client: ssh exited with exit code 2 pdsh@oleg405-client: oleg405-server: ssh exited with exit code 2 pdsh@oleg405-client: oleg405-client: ssh exited with exit code 2 pdsh@oleg405-client: oleg405-server: ssh exited with exit code 2 PASS 106 (1260s) Stopping clients: oleg405-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg405-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg405-client: oleg405-server: ssh exited with exit code 2 oleg405-server: oleg405-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: detected 4 online CPUs by sysfs oleg405-server: MODOPTS_LIBCFS= oleg405-server: Force libcfs to create 2 CPU partitions oleg405-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg405-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1335 sec ========== 08:10:24 (1746792624) === conf-sanity: start cleanup 08:10:25 (1746792625) === === conf-sanity: finish cleanup 08:10:25 (1746792625) ===