-----============= acceptance-small: conf-sanity ============----- Tue Apr 8 23:44:14 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg333-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg333-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg333-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg333-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg333-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg333-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg333-server oleg333-server: oleg333-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory loading modules on: 'oleg333-server' oleg333-server: oleg333-server.virtnet: executing load_modules_local oleg333-server: Loading modules from /home/green/git/lustre-release/lustre oleg333-server: detected 4 online CPUs by sysfs oleg333-server: MODOPTS_LIBCFS= oleg333-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg333-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg333-server: oleg333-server.virtnet: executing set_default_debug -1 all pdsh@oleg333-client: oleg333-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg333-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg333-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg333-server: oleg333-server.virtnet: executing set_default_debug -1 all pdsh@oleg333-client: oleg333-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg333-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg333-server: oleg333-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg333-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg333-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg333-server stop mds service on oleg333-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg333-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 23:45:10 (1744170310) Stopping clients: oleg333-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg333-client.virtnet /mnt/lustre2 (opts:-f) oleg333-server: oleg333-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg333-server' oleg333-server: oleg333-server.virtnet: executing load_modules_local oleg333-server: Loading modules from /home/green/git/lustre-release/lustre oleg333-server: detected 4 online CPUs by sysfs oleg333-server: MODOPTS_LIBCFS= oleg333-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg333-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg333-server: oleg333-server.virtnet: executing set_default_debug -1 all pdsh@oleg333-client: oleg333-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg333-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg333-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg333-server: oleg333-server.virtnet: executing set_default_debug -1 all pdsh@oleg333-client: oleg333-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg333-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg333-client.virtnet: -o user_xattr,flock 192.168.203.133@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86890 331 86559 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32587 331 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 4348 (time 1744170345.37 total 10.00 last 434.76) - open/close 8979 (time 1744170355.37 total 20.00 last 463.08) - open/close 13783 (time 1744170365.37 total 30.00 last 480.28) - open/close 17889 (time 1744170375.37 total 40.00 last 410.60) - open/close 20000 (time 1744170380.75 total 45.38 last 392.78) - open/close 24197 (time 1744170390.75 total 55.38 last 419.64) - open/close 28428 (time 1744170400.75 total 65.38 last 423.09) - open/close 30000 (time 1744170404.41 total 69.04 last 429.35) - open/close 34471 (time 1744170414.41 total 79.04 last 447.06) - open/close 39113 (time 1744170424.41 total 89.04 last 464.16) - open/close 43626 (time 1744170434.41 total 99.04 last 451.27) - open/close 48324 (time 1744170444.41 total 109.04 last 469.78) - open/close 50000 (time 1744170447.38 total 112.01 last 565.43) - open/close 54870 (time 1744170457.38 total 122.01 last 486.94) - open/close 60000 (time 1744170467.20 total 131.83 last 522.51) total: 64768 open/close in 141.33 seconds: 458.27 ops/second - /unlink 7262 (time 1744170487.59 total 10.00 last 726.18) - /unlink 10000 (time 1744170491.55 total 13.96 last 691.24) - /unlink 17412 (time 1744170501.55 total 23.96 last 741.15) - /unlink 20000 (time 1744170505.15 total 27.56 last 719.64) - /unlink 27109 (time 1744170515.15 total 37.56 last 710.89) - /unlink 30000 (time 1744170519.29 total 41.70 last 698.24) - /unlink 37505 (time 1744170529.29 total 51.70 last 750.43) - /unlink 40000 (time 1744170532.99 total 55.39 last 675.30) - /unlink 47854 (time 1744170542.99 total 65.39 last 785.40) - /unlink 50000 (time 1744170545.43 total 67.84 last 877.02) - /unlink 58822 (time 1744170555.43 total 77.84 last 882.13) total: 64768 /unlink in 84.26 seconds: 768.68 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5222 (time 1744170588.49 total 10.00 last 522.12) - open/close 10000 (time 1744170598.06 total 19.57 last 499.37) - open/close 14928 (time 1744170608.06 total 29.57 last 492.73) - open/close 20000 (time 1744170617.85 total 39.36 last 518.11) - open/close 25289 (time 1744170627.85 total 49.36 last 528.86) - open/close 30000 (time 1744170636.76 total 58.27 last 528.68) - open/close 35297 (time 1744170646.76 total 68.27 last 529.64) - open/close 40000 (time 1744170655.65 total 77.16 last 529.14) - open/close 45371 (time 1744170665.65 total 87.16 last 537.06) - open/close 50000 (time 1744170674.68 total 96.19 last 512.53) - open/close 55242 (time 1744170684.69 total 106.20 last 524.12) - open/close 60000 (time 1744170693.72 total 115.23 last 526.71) total: 64768 open/close in 124.18 seconds: 521.56 ops/second - /unlink 8926 (time 1744170713.51 total 10.00 last 892.53) - /unlink 18119 (time 1744170723.51 total 20.00 last 919.24) - /unlink 20000 (time 1744170725.53 total 22.02 last 930.04) - /unlink 28635 (time 1744170735.53 total 32.02 last 863.49) - /unlink 37530 (time 1744170745.53 total 42.02 last 889.47) - /unlink 40000 (time 1744170748.48 total 44.97 last 838.51) - /unlink 49142 (time 1744170758.48 total 54.97 last 914.16) - /unlink 58633 (time 1744170768.48 total 64.97 last 949.08) total: 64768 /unlink in 71.51 seconds: 905.74 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5662 (time 1744170814.46 total 10.00 last 566.11) - open/close 10000 (time 1744170822.36 total 17.90 last 549.40) - open/close 15655 (time 1744170832.36 total 27.90 last 565.49) - open/close 20000 (time 1744170840.17 total 35.71 last 556.22) - open/close 25504 (time 1744170850.17 total 45.71 last 550.31) - open/close 30000 (time 1744170858.25 total 53.78 last 556.98) - open/close 35536 (time 1744170868.25 total 63.78 last 553.57) - open/close 40000 (time 1744170876.21 total 71.74 last 560.70) - open/close 45348 (time 1744170886.21 total 81.75 last 534.74) - open/close 50000 (time 1744170895.24 total 90.78 last 514.90) - open/close 55437 (time 1744170905.24 total 100.78 last 543.67) - open/close 60000 (time 1744170913.75 total 109.28 last 536.74) total: 64768 open/close in 118.27 seconds: 547.62 ops/second - /unlink 9475 (time 1744170933.57 total 10.00 last 947.47) - /unlink 18861 (time 1744170943.57 total 20.00 last 938.55) - /unlink 28337 (time 1744170953.57 total 30.00 last 947.50) - /unlink 37338 (time 1744170963.57 total 40.00 last 900.06) - /unlink 40000 (time 1744170966.75 total 43.18 last 838.06) - /unlink 48959 (time 1744170976.75 total 53.18 last 895.90) - /unlink 58112 (time 1744170986.75 total 63.18 last 915.29) total: 64768 /unlink in 70.41 seconds: 919.88 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5538 (time 1744171034.81 total 10.00 last 553.72) - open/close 10000 (time 1744171042.85 total 18.05 last 554.69) - open/close 15457 (time 1744171052.85 total 28.05 last 545.67) - open/close 20000 (time 1744171061.19 total 36.38 last 544.93) - open/close 25151 (time 1744171071.19 total 46.38 last 515.04) - open/close 29579 (time 1744171081.20 total 56.39 last 442.66) - open/close 33167 (time 1744171091.20 total 66.39 last 358.79) - open/close 36760 (time 1744171101.20 total 76.39 last 359.26) - open/close 39991 (time 1744171111.20 total 86.39 last 323.06) - open/close 44282 (time 1744171121.20 total 96.39 last 429.05) - open/close 49285 (time 1744171131.20 total 106.39 last 500.22) - open/close 54291 (time 1744171141.20 total 116.39 last 500.59) - open/close 58304 (time 1744171151.20 total 126.39 last 401.23) - open/close 60000 (time 1744171157.27 total 132.46 last 279.63) - open/close 64431 (time 1744171167.27 total 142.46 last 443.01) total: 64768 open/close in 143.30 seconds: 451.98 ops/second - /unlink 9207 (time 1744171178.95 total 10.00 last 920.66) - /unlink 18012 (time 1744171188.96 total 20.00 last 880.39) - /unlink 20000 (time 1744171191.03 total 22.08 last 957.62) - /unlink 28432 (time 1744171201.03 total 32.08 last 843.10) - /unlink 35610 (time 1744171211.03 total 42.08 last 717.66) - /unlink 40000 (time 1744171217.61 total 48.65 last 668.05) - /unlink 48893 (time 1744171227.61 total 58.65 last 889.23) - /unlink 56510 (time 1744171237.61 total 68.65 last 761.65) - /unlink 60000 (time 1744171241.97 total 73.01 last 800.74) total: 64768 /unlink in 78.16 seconds: 828.62 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 3945 (time 1744171280.78 total 10.00 last 394.46) - open/close 8242 (time 1744171290.78 total 20.00 last 429.60) - open/close 10000 (time 1744171294.65 total 23.88 last 453.56) - open/close 14515 (time 1744171304.66 total 33.88 last 451.39) - open/close 18325 (time 1744171314.66 total 43.88 last 380.88) - open/close 20000 (time 1744171319.87 total 49.10 last 321.32) - open/close 24308 (time 1744171329.87 total 59.10 last 430.79) - open/close 29151 (time 1744171339.87 total 69.10 last 484.28) - open/close 33567 (time 1744171349.87 total 79.10 last 441.60) - open/close 37210 (time 1744171359.87 total 89.10 last 364.25) - open/close 40000 (time 1744171367.25 total 96.47 last 378.51) - open/close 43528 (time 1744171377.25 total 106.47 last 352.77) - open/close 46980 (time 1744171387.25 total 116.47 last 345.20) - open/close 50000 (time 1744171396.45 total 125.68 last 328.11) - open/close 54134 (time 1744171406.45 total 135.68 last 413.38) - open/close 57321 (time 1744171416.45 total 145.68 last 318.62) - open/close 59924 (time 1744171426.45 total 155.68 last 260.27) - open/close 62640 (time 1744171436.46 total 165.68 last 271.56) total: 64768 open/close in 169.52 seconds: 382.07 ops/second - /unlink 6024 (time 1744171451.38 total 10.00 last 602.40) - /unlink 10000 (time 1744171458.82 total 17.44 last 534.44) - /unlink 16033 (time 1744171468.82 total 27.44 last 603.28) - /unlink 20000 (time 1744171476.96 total 35.59 last 486.90) - /unlink 25169 (time 1744171486.96 total 45.59 last 516.85) - /unlink 30000 (time 1744171495.74 total 54.36 last 550.56) - /unlink 35178 (time 1744171505.74 total 64.36 last 517.77) - /unlink 40000 (time 1744171514.68 total 73.31 last 539.10) - /unlink 45783 (time 1744171524.68 total 83.31 last 578.29) - /unlink 50000 (time 1744171532.88 total 91.51 last 514.46) - /unlink 55081 (time 1744171542.88 total 101.51 last 508.04) - /unlink 60000 (time 1744171552.01 total 110.63 last 538.91) total: 64768 /unlink in 117.62 seconds: 550.64 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg333-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg333-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg333-server stop mds service on oleg333-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg333-server unloading modules via unload_modules_local on: 'oleg333-server' oleg333-server: oleg333-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg333-client: oleg333-client: ssh exited with exit code 2 pdsh@oleg333-client: oleg333-server: ssh exited with exit code 2 pdsh@oleg333-client: oleg333-client: ssh exited with exit code 2 pdsh@oleg333-client: oleg333-server: ssh exited with exit code 2 PASS 106 (1287s) Stopping clients: oleg333-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg333-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg333-client: oleg333-server: ssh exited with exit code 2 oleg333-server: oleg333-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg333-server' oleg333-server: oleg333-server.virtnet: executing load_modules_local oleg333-server: Loading modules from /home/green/git/lustre-release/lustre oleg333-server: detected 4 online CPUs by sysfs oleg333-server: MODOPTS_LIBCFS= oleg333-server: Force libcfs to create 2 CPU partitions oleg333-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg333-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1366 sec ========== 00:07:00 (1744171620) === conf-sanity: start cleanup 00:07:00 (1744171620) === === conf-sanity: finish cleanup 00:07:01 (1744171621) ===