-----============= acceptance-small: conf-sanity ============----- Mon Mar 31 16:51:12 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg141-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg141-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg141-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg141-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg141-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg141-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg141-server oleg141-server: oleg141-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory loading modules on: 'oleg141-server' oleg141-server: oleg141-server.virtnet: executing load_modules_local oleg141-server: Loading modules from /home/green/git/lustre-release/lustre oleg141-server: detected 4 online CPUs by sysfs oleg141-server: MODOPTS_LIBCFS= oleg141-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg141-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg141-server: oleg141-server.virtnet: executing set_default_debug -1 all pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg141-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg141-server: oleg141-server.virtnet: executing set_default_debug -1 all pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg141-server: oleg141-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg141-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg141-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg141-server stop mds service on oleg141-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg141-server == conf-sanity test 106: check osp llog processing when catalog is wrapped ========================================================== 16:52:10 (1743454330) Stopping clients: oleg141-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg141-client.virtnet /mnt/lustre2 (opts:-f) oleg141-server: oleg141-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg141-server' oleg141-server: oleg141-server.virtnet: executing load_modules_local oleg141-server: Loading modules from /home/green/git/lustre-release/lustre oleg141-server: detected 4 online CPUs by sysfs oleg141-server: MODOPTS_LIBCFS= oleg141-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg141-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg141-server: oleg141-server.virtnet: executing set_default_debug -1 all pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg141-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg141-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg141-server: oleg141-server.virtnet: executing set_default_debug -1 all pdsh@oleg141-client: oleg141-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg141-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg141-client.virtnet: -o user_xattr,flock 192.168.201.141@tcp:/lustre /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 86842 329 86513 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 32582 326 32256 2% /mnt/lustre[OST:0] filesystem_summary: 32585 329 32256 2% /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff fail_loc=0x1312 fail_val=5 - open/close 4289 (time 1743454366.06 total 10.00 last 428.85) - open/close 9764 (time 1743454376.06 total 20.00 last 547.42) - open/close 14705 (time 1743454386.06 total 30.00 last 494.06) - open/close 18800 (time 1743454396.06 total 40.00 last 409.50) - open/close 20000 (time 1743454398.53 total 42.47 last 485.76) - open/close 23711 (time 1743454408.53 total 52.48 last 371.03) - open/close 28754 (time 1743454418.53 total 62.48 last 504.27) - open/close 30000 (time 1743454420.56 total 64.51 last 613.42) - open/close 35889 (time 1743454430.56 total 74.51 last 588.90) - open/close 40000 (time 1743454438.03 total 81.98 last 550.45) - open/close 45719 (time 1743454448.03 total 91.98 last 571.84) - open/close 50000 (time 1743454455.04 total 98.98 last 611.01) - open/close 55293 (time 1743454465.04 total 108.98 last 529.29) - open/close 60000 (time 1743454474.95 total 118.90 last 474.78) - open/close 64521 (time 1743454484.95 total 128.90 last 452.05) total: 64768 open/close in 129.45 seconds: 500.33 ops/second - /unlink 9625 (time 1743454496.31 total 10.00 last 962.46) - /unlink 19015 (time 1743454506.31 total 20.00 last 938.89) - /unlink 26783 (time 1743454516.31 total 30.00 last 776.76) - /unlink 30000 (time 1743454521.03 total 34.73 last 681.07) - /unlink 37163 (time 1743454531.03 total 44.73 last 716.28) - /unlink 40000 (time 1743454535.19 total 48.88 last 683.21) - /unlink 46605 (time 1743454545.19 total 58.88 last 660.47) - /unlink 50000 (time 1743454550.30 total 64.00 last 663.40) - /unlink 56468 (time 1743454560.31 total 74.00 last 646.64) - /unlink 60000 (time 1743454565.54 total 79.23 last 675.54) total: 64768 /unlink in 86.07 seconds: 752.50 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 4704 (time 1743454598.82 total 10.00 last 470.36) - open/close 9292 (time 1743454608.82 total 20.00 last 458.77) - open/close 14196 (time 1743454618.82 total 30.00 last 490.34) - open/close 18825 (time 1743454628.82 total 40.00 last 462.82) - open/close 20000 (time 1743454631.41 total 42.60 last 453.12) - open/close 24412 (time 1743454641.42 total 52.60 last 441.13) - open/close 28697 (time 1743454651.42 total 62.60 last 428.43) - open/close 30000 (time 1743454654.53 total 65.71 last 418.73) - open/close 34387 (time 1743454664.53 total 75.71 last 438.65) - open/close 38523 (time 1743454674.53 total 85.71 last 413.55) - open/close 40000 (time 1743454677.97 total 89.16 last 429.18) - open/close 44407 (time 1743454687.98 total 99.16 last 440.58) - open/close 48531 (time 1743454697.98 total 109.16 last 412.27) - open/close 50000 (time 1743454701.53 total 112.72 last 413.34) - open/close 54415 (time 1743454711.53 total 122.72 last 441.44) - open/close 59091 (time 1743454721.54 total 132.72 last 467.50) - open/close 60000 (time 1743454723.55 total 134.73 last 452.25) - open/close 64509 (time 1743454733.55 total 144.73 last 450.89) total: 64768 open/close in 145.38 seconds: 445.50 ops/second - /unlink 7669 (time 1743454745.12 total 10.00 last 766.85) - /unlink 10000 (time 1743454748.34 total 13.22 last 723.50) - /unlink 17304 (time 1743454758.34 total 23.22 last 730.32) - /unlink 20000 (time 1743454761.98 total 26.86 last 740.77) - /unlink 27157 (time 1743454771.98 total 36.86 last 715.61) - /unlink 30000 (time 1743454775.81 total 40.69 last 743.30) - /unlink 37177 (time 1743454785.81 total 50.69 last 717.66) - /unlink 40000 (time 1743454789.63 total 54.51 last 739.03) - /unlink 47371 (time 1743454799.63 total 64.51 last 736.91) - /unlink 50000 (time 1743454803.35 total 68.24 last 705.90) - /unlink 57253 (time 1743454813.35 total 78.24 last 725.19) - /unlink 60000 (time 1743454817.19 total 82.08 last 715.47) total: 64768 /unlink in 88.53 seconds: 731.58 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 4545 (time 1743454854.82 total 10.00 last 454.43) - open/close 9301 (time 1743454864.82 total 20.00 last 475.56) - open/close 13975 (time 1743454874.82 total 30.00 last 467.38) - open/close 19063 (time 1743454884.82 total 40.00 last 508.73) - open/close 24340 (time 1743454894.82 total 50.00 last 527.65) - open/close 29526 (time 1743454904.82 total 60.01 last 518.54) - open/close 34800 (time 1743454914.82 total 70.01 last 527.31) - open/close 39636 (time 1743454924.82 total 80.01 last 483.59) - open/close 44654 (time 1743454934.82 total 90.01 last 501.75) - open/close 49658 (time 1743454944.83 total 100.01 last 500.37) - open/close 54681 (time 1743454954.83 total 110.01 last 502.25) - open/close 59662 (time 1743454964.83 total 120.01 last 498.05) - open/close 64497 (time 1743454974.83 total 130.01 last 483.48) total: 64768 open/close in 130.62 seconds: 495.84 ops/second - /unlink 8427 (time 1743454986.30 total 10.00 last 842.61) - /unlink 16778 (time 1743454996.31 total 20.00 last 834.98) - /unlink 20000 (time 1743455000.26 total 23.96 last 815.07) - /unlink 28481 (time 1743455010.26 total 33.96 last 848.03) - /unlink 37426 (time 1743455020.26 total 43.96 last 894.43) - /unlink 40000 (time 1743455023.05 total 46.75 last 922.69) - /unlink 49111 (time 1743455033.05 total 56.75 last 911.07) - /unlink 57971 (time 1743455043.05 total 66.75 last 885.93) - /unlink 60000 (time 1743455045.28 total 68.98 last 910.65) total: 64768 /unlink in 74.08 seconds: 874.33 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 5435 (time 1743455085.50 total 10.00 last 543.43) - open/close 10000 (time 1743455094.03 total 18.53 last 535.46) - open/close 14749 (time 1743455104.03 total 28.53 last 474.86) - open/close 19606 (time 1743455114.03 total 38.53 last 485.62) - open/close 24684 (time 1743455124.05 total 48.55 last 506.69) - open/close 29822 (time 1743455134.05 total 58.55 last 513.78) - open/close 34853 (time 1743455144.05 total 68.55 last 503.02) - open/close 38764 (time 1743455154.06 total 78.56 last 390.99) - open/close 40000 (time 1743455156.62 total 81.12 last 482.80) - open/close 43670 (time 1743455166.62 total 91.12 last 366.88) - open/close 48579 (time 1743455176.62 total 101.12 last 490.85) - open/close 50000 (time 1743455179.55 total 104.05 last 484.81) - open/close 54994 (time 1743455189.55 total 114.05 last 499.31) - open/close 60000 (time 1743455199.34 total 123.84 last 511.52) total: 64768 open/close in 132.91 seconds: 487.32 ops/second - /unlink 7199 (time 1743455219.28 total 10.00 last 719.88) - /unlink 10000 (time 1743455222.54 total 13.26 last 858.95) - /unlink 19111 (time 1743455232.54 total 23.26 last 911.03) - /unlink 27042 (time 1743455242.54 total 33.26 last 793.04) - /unlink 30000 (time 1743455246.93 total 37.64 last 675.03) - /unlink 38018 (time 1743455256.93 total 47.65 last 801.70) - /unlink 40000 (time 1743455259.58 total 50.29 last 748.60) - /unlink 48670 (time 1743455269.58 total 60.29 last 866.93) - /unlink 57045 (time 1743455279.58 total 70.30 last 837.41) - /unlink 60000 (time 1743455283.38 total 74.09 last 777.85) total: 64768 /unlink in 79.28 seconds: 816.90 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete - open/close 4832 (time 1743455319.72 total 10.00 last 483.15) - open/close 9609 (time 1743455329.72 total 20.00 last 477.63) - open/close 14291 (time 1743455339.72 total 30.00 last 468.16) - open/close 19091 (time 1743455349.72 total 40.00 last 479.93) - open/close 24011 (time 1743455359.73 total 50.01 last 491.97) - open/close 28601 (time 1743455369.73 total 60.01 last 458.98) - open/close 30000 (time 1743455372.84 total 63.12 last 448.54) - open/close 34832 (time 1743455382.85 total 73.13 last 483.11) - open/close 38242 (time 1743455392.85 total 83.13 last 340.94) - open/close 40000 (time 1743455396.67 total 86.95 last 459.55) - open/close 43672 (time 1743455406.68 total 96.96 last 367.15) - open/close 48130 (time 1743455416.68 total 106.96 last 445.64) - open/close 50000 (time 1743455422.42 total 112.70 last 325.87) - open/close 54848 (time 1743455432.42 total 122.70 last 484.74) - open/close 59787 (time 1743455442.42 total 132.70 last 493.84) - open/close 64530 (time 1743455452.42 total 142.70 last 474.25) total: 64768 open/close in 143.16 seconds: 452.42 ops/second - /unlink 9356 (time 1743455463.69 total 10.00 last 935.52) - /unlink 17738 (time 1743455473.69 total 20.00 last 838.12) - /unlink 20000 (time 1743455476.38 total 22.69 last 840.15) - /unlink 27318 (time 1743455486.38 total 32.69 last 731.76) - /unlink 30000 (time 1743455489.65 total 35.97 last 818.90) - /unlink 38907 (time 1743455499.66 total 45.97 last 890.66) - /unlink 47650 (time 1743455509.66 total 55.97 last 874.26) - /unlink 50000 (time 1743455512.77 total 59.08 last 755.40) - /unlink 56424 (time 1743455522.77 total 69.08 last 642.33) - /unlink 60000 (time 1743455526.31 total 72.62 last 1010.85) total: 64768 /unlink in 77.37 seconds: 837.17 ops/second sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0 fail_val=0 umount lustre on /mnt/lustre..... Stopping client oleg141-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg141-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg141-server stop mds service on oleg141-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg141-server unloading modules via unload_modules_local on: 'oleg141-server' oleg141-server: oleg141-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg141-client: oleg141-client: ssh exited with exit code 2 pdsh@oleg141-client: oleg141-server: ssh exited with exit code 2 pdsh@oleg141-client: oleg141-client: ssh exited with exit code 2 pdsh@oleg141-client: oleg141-server: ssh exited with exit code 2 PASS 106 (1245s) Stopping clients: oleg141-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg141-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg141-client: oleg141-server: ssh exited with exit code 2 oleg141-server: oleg141-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg141-server' oleg141-server: oleg141-server.virtnet: executing load_modules_local oleg141-server: Loading modules from /home/green/git/lustre-release/lustre oleg141-server: detected 4 online CPUs by sysfs oleg141-server: MODOPTS_LIBCFS= oleg141-server: Force libcfs to create 2 CPU partitions oleg141-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg141-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg141-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1321 sec ========== 17:13:13 (1743455593) === conf-sanity: start cleanup 17:13:13 (1743455593) === === conf-sanity: finish cleanup 17:13:14 (1743455594) ===