== conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 12:49:51 (1721407791) Stopping clients: oleg330-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg330-client.virtnet /mnt/lustre2 (opts:-f) oleg330-server: oleg330-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory loading modules on: 'oleg330-server' oleg330-server: oleg330-server.virtnet: executing load_modules_local oleg330-server: Loading modules from /home/green/git/lustre-release/lustre oleg330-server: detected 4 online CPUs by sysfs oleg330-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg330-server' oleg330-server: oleg330-server.virtnet: executing load_modules_local oleg330-server: Loading modules from /home/green/git/lustre-release/lustre oleg330-server: detected 4 online CPUs by sysfs oleg330-server: Force libcfs to create 2 CPU partitions fail_loc=0x00001e0 start mds service on oleg330-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg330-server: oleg330-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg330-client: oleg330-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg330-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg330-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg330-server: oleg330-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg330-client: oleg330-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg330-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg330-client.virtnet: -o user_xattr,flock oleg330-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 4096 (time 1721407840.68 total 10.00 last 409.44) - open/close 8339 (time 1721407850.68 total 20.01 last 424.24) total: 10000 open/close in 23.47 seconds: 426.03 ops/second - unlinked 0 (time 1721407855 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 4040 (time 1721407877.09 total 10.00 last 403.97) - open/close 7376 (time 1721407887.09 total 20.00 last 333.59) total: 10000 open/close in 27.41 seconds: 364.79 ops/second - unlinked 0 (time 1721407895 ; total 0 ; last 0) total: 10000 unlinks in 18 seconds: 555.555542 unlinks/second - open/close 2748 (time 1721407924.41 total 10.00 last 274.72) - open/close 5477 (time 1721407934.42 total 20.00 last 272.89) - open/close 8604 (time 1721407944.42 total 30.01 last 312.62) total: 10000 open/close in 34.53 seconds: 289.58 ops/second - unlinked 0 (time 1721407950 ; total 0 ; last 0) total: 10000 unlinks in 16 seconds: 625.000000 unlinks/second - open/close 4279 (time 1721407977.93 total 10.00 last 427.83) - open/close 7341 (time 1721407987.93 total 20.00 last 306.16) total: 10000 open/close in 24.88 seconds: 401.92 ops/second - unlinked 0 (time 1721407993 ; total 0 ; last 0) total: 10000 unlinks in 32 seconds: 312.500000 unlinks/second - open/close 1858 (time 1721408036.71 total 10.00 last 185.74) - open/close 3170 (time 1721408046.72 total 20.02 last 131.01) - open/close 4592 (time 1721408056.73 total 30.02 last 142.18) - open/close 6055 (time 1721408066.73 total 40.02 last 146.22) - open/close 7609 (time 1721408076.74 total 50.03 last 155.31) - open/close 8810 (time 1721408086.74 total 60.03 last 120.04) - open/close 9878 (time 1721408096.75 total 70.05 last 106.67) total: 10000 open/close in 71.25 seconds: 140.35 ops/second - unlinked 0 (time 1721408100 ; total 0 ; last 0) total: 10000 unlinks in 41 seconds: 243.902435 unlinks/second - open/close 1859 (time 1721408153.47 total 10.01 last 185.77) - open/close 3080 (time 1721408163.47 total 20.01 last 122.04) - open/close 4235 (time 1721408173.47 total 30.01 last 115.47) - open/close 5487 (time 1721408183.48 total 40.02 last 125.08) - open/close 7117 (time 1721408193.49 total 50.03 last 162.98) total: 10000 open/close in 57.62 seconds: 173.56 ops/second - unlinked 0 (time 1721408202 ; total 0 ; last 0) total: 10000 unlinks in 18 seconds: 555.555542 unlinks/second - open/close 1520 (time 1721408232.31 total 10.00 last 151.97) - open/close 2651 (time 1721408242.31 total 20.00 last 113.09) - open/close 3783 (time 1721408252.32 total 30.00 last 113.18) - open/close 5041 (time 1721408262.33 total 40.02 last 125.60) - open/close 6178 (time 1721408272.34 total 50.03 last 113.65) - open/close 7408 (time 1721408282.34 total 60.03 last 122.91) - open/close 8512 (time 1721408292.35 total 70.04 last 110.34) - open/close 9672 (time 1721408302.35 total 80.04 last 115.94) total: 10000 open/close in 82.69 seconds: 120.93 ops/second - unlinked 0 (time 1721408306 ; total 0 ; last 0) total: 10000 unlinks in 47 seconds: 212.765961 unlinks/second - open/close 1452 (time 1721408365.49 total 10.00 last 145.16) - open/close 2776 (time 1721408375.49 total 20.00 last 132.40) - open/close 4307 (time 1721408385.49 total 30.01 last 153.07) - open/close 5795 (time 1721408395.49 total 40.01 last 148.80) - open/close 8004 (time 1721408405.49 total 50.01 last 220.88) total: 10000 open/close in 57.89 seconds: 172.73 ops/second - unlinked 0 (time 1721408414 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second - open/close 1529 (time 1721408445.41 total 10.00 last 152.87) - open/close 2749 (time 1721408455.42 total 20.01 last 121.87) - open/close 3982 (time 1721408465.42 total 30.02 last 123.26) - open/close 5023 (time 1721408475.42 total 40.02 last 104.09) - open/close 6380 (time 1721408485.43 total 50.02 last 135.61) - open/close 7624 (time 1721408495.43 total 60.02 last 124.39) - open/close 8668 (time 1721408505.43 total 70.03 last 104.39) - open/close 9745 (time 1721408515.43 total 80.03 last 107.67) total: 10000 open/close in 82.47 seconds: 121.26 ops/second - unlinked 0 (time 1721408520 ; total 0 ; last 0) total: 10000 unlinks in 47 seconds: 212.765961 unlinks/second - open/close 1332 (time 1721408579.04 total 10.00 last 133.18) - open/close 2665 (time 1721408589.04 total 20.01 last 133.24) - open/close 4058 (time 1721408599.04 total 30.01 last 139.30) - open/close 5730 (time 1721408609.04 total 40.01 last 167.12) - open/close 6947 (time 1721408619.05 total 50.02 last 121.64) - open/close 8101 (time 1721408629.05 total 60.02 last 115.37) - open/close 9304 (time 1721408639.05 total 70.02 last 120.29) total: 10000 open/close in 75.22 seconds: 132.94 ops/second - unlinked 0 (time 1721408645 ; total 0 ; last 0) total: 10000 unlinks in 31 seconds: 322.580658 unlinks/second - open/close 1574 (time 1721408687.32 total 10.01 last 157.31) - open/close 2631 (time 1721408697.32 total 20.01 last 105.66) - open/close 4012 (time 1721408707.32 total 30.01 last 138.06) - open/close 5571 (time 1721408717.32 total 40.01 last 155.88) - open/close 7780 (time 1721408727.33 total 50.01 last 220.88) total: 10000 open/close in 57.40 seconds: 174.23 ops/second - unlinked 0 (time 1721408735 ; total 0 ; last 0) total: 10000 unlinks in 19 seconds: 526.315796 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5760 375808 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 3072 377856 1% /mnt/lustre[OST:0] filesystem_summary: 382976 3072 377856 1% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.227196 s, 4.6 MB/s fail_loc=0x80000204 Failing ost1 on oleg330-server Stopping /mnt/lustre-ost1 (opts:) on oleg330-server reboot facets: ost1 Failover ost1 to oleg330-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg330-server: oleg330-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg330-client: oleg330-server: ssh exited with exit code 1 Started lustre-OST0000 oleg330-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1721408783.50 total 5.08 last 0.00) total: 100 open/close in 5.57 seconds: 17.95 ops/second umount lustre on /mnt/lustre..... Stopping client oleg330-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg330-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg330-server stop mds service on oleg330-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg330-server LNET ready to unload unloading modules on: 'oleg330-server' oleg330-server: oleg330-server.virtnet: executing unload_modules_local oleg330-server: LNET ready to unload modules unloaded.