== conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 10:20:09 (1736263209) Stopping clients: oleg206-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg206-client.virtnet /mnt/lustre2 (opts:-f) oleg206-server: oleg206-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory loading modules on: 'oleg206-server' oleg206-server: oleg206-server.virtnet: executing load_modules_local oleg206-server: Loading modules from /home/green/git/lustre-release/lustre oleg206-server: detected 4 online CPUs by sysfs oleg206-server: Force libcfs to create 2 CPU partitions oleg206-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg206-server' oleg206-server: oleg206-server.virtnet: executing load_modules_local oleg206-server: Loading modules from /home/green/git/lustre-release/lustre oleg206-server: detected 4 online CPUs by sysfs oleg206-server: Force libcfs to create 2 CPU partitions fail_loc=0x00001e0 start mds service on oleg206-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg206-server: oleg206-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg206-client: oleg206-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg206-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg206-server: oleg206-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg206-client: oleg206-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg206-client.virtnet: -o user_xattr,flock oleg206-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 1977 (time 1736263275.78 total 10.01 last 197.59) - open/close 3523 (time 1736263285.78 total 20.01 last 154.58) - open/close 4977 (time 1736263295.79 total 30.01 last 145.29) - open/close 6478 (time 1736263305.79 total 40.02 last 150.04) - open/close 8334 (time 1736263315.80 total 50.02 last 185.50) total: 10000 open/close in 58.68 seconds: 170.41 ops/second - unlinked 0 (time 1736263325 ; total 0 ; last 0) total: 10000 unlinks in 27 seconds: 370.370361 unlinks/second - open/close 2434 (time 1736263363.40 total 10.00 last 243.32) - open/close 4737 (time 1736263373.41 total 20.01 last 230.22) - open/close 6621 (time 1736263383.41 total 30.01 last 188.36) - open/close 8348 (time 1736263393.41 total 40.01 last 172.67) total: 10000 open/close in 48.99 seconds: 204.12 ops/second - unlinked 0 (time 1736263403 ; total 0 ; last 0) total: 10000 unlinks in 31 seconds: 322.580658 unlinks/second - open/close 2392 (time 1736263445.70 total 10.00 last 239.19) - open/close 4686 (time 1736263455.70 total 20.00 last 229.30) - open/close 6605 (time 1736263465.70 total 30.01 last 191.84) - open/close 8654 (time 1736263475.71 total 40.01 last 204.81) total: 10000 open/close in 46.66 seconds: 214.32 ops/second - unlinked 0 (time 1736263483 ; total 0 ; last 0) total: 10000 unlinks in 31 seconds: 322.580658 unlinks/second - open/close 1891 (time 1736263525.66 total 10.00 last 189.06) - open/close 4267 (time 1736263535.66 total 20.01 last 237.53) - open/close 6543 (time 1736263545.67 total 30.01 last 227.49) - open/close 8411 (time 1736263555.67 total 40.01 last 186.71) total: 10000 open/close in 47.49 seconds: 210.55 ops/second - unlinked 0 (time 1736263564 ; total 0 ; last 0) total: 10000 unlinks in 27 seconds: 370.370361 unlinks/second - open/close 2707 (time 1736263602.95 total 10.00 last 270.63) - open/close 5646 (time 1736263612.95 total 20.00 last 293.85) - open/close 8233 (time 1736263622.95 total 30.01 last 258.56) total: 10000 open/close in 39.19 seconds: 255.15 ops/second - unlinked 0 (time 1736263633 ; total 0 ; last 0) total: 10000 unlinks in 26 seconds: 384.615387 unlinks/second - open/close 3325 (time 1736263670.51 total 10.00 last 332.43) - open/close 6402 (time 1736263680.51 total 20.00 last 307.70) - open/close 9158 (time 1736263690.51 total 30.00 last 275.58) total: 10000 open/close in 33.37 seconds: 299.67 ops/second - unlinked 0 (time 1736263694 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second - open/close 2143 (time 1736263725.71 total 10.00 last 214.22) - open/close 4396 (time 1736263735.71 total 20.01 last 225.20) - open/close 6763 (time 1736263745.71 total 30.01 last 236.69) - open/close 8915 (time 1736263755.71 total 40.01 last 215.16) total: 10000 open/close in 44.69 seconds: 223.77 ops/second - unlinked 0 (time 1736263761 ; total 0 ; last 0) total: 10000 unlinks in 23 seconds: 434.782623 unlinks/second - open/close 2411 (time 1736263795.90 total 10.00 last 241.07) - open/close 5277 (time 1736263805.90 total 20.00 last 286.59) - open/close 8671 (time 1736263815.90 total 30.00 last 339.38) total: 10000 open/close in 34.40 seconds: 290.72 ops/second - unlinked 0 (time 1736263821 ; total 0 ; last 0) total: 10000 unlinks in 18 seconds: 555.555542 unlinks/second - open/close 3243 (time 1736263850.81 total 10.00 last 324.20) - open/close 5781 (time 1736263860.81 total 20.00 last 253.77) - open/close 8829 (time 1736263870.81 total 30.01 last 304.76) total: 10000 open/close in 33.40 seconds: 299.40 ops/second - unlinked 0 (time 1736263875 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 2825 (time 1736263900.97 total 10.00 last 282.46) - open/close 6157 (time 1736263910.97 total 20.00 last 333.11) - open/close 8851 (time 1736263920.98 total 30.01 last 269.33) total: 10000 open/close in 34.05 seconds: 293.73 ops/second - unlinked 0 (time 1736263926 ; total 0 ; last 0) total: 10000 unlinks in 18 seconds: 555.555542 unlinks/second - open/close 2926 (time 1736263955.44 total 10.00 last 292.50) - open/close 6295 (time 1736263965.44 total 20.00 last 336.87) - open/close 9574 (time 1736263975.44 total 30.01 last 327.87) total: 10000 open/close in 31.09 seconds: 321.69 ops/second - unlinked 0 (time 1736263977 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 6144 375424 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 4096 376832 2% /mnt/lustre[OST:0] filesystem_summary: 382976 4096 376832 2% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.225404 s, 4.7 MB/s fail_loc=0x80000204 Failing ost1 on oleg206-server Stopping /mnt/lustre-ost1 (opts:) on oleg206-server reboot facets: ost1 Failover ost1 to oleg206-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg206-server: oleg206-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg206-client: oleg206-server: ssh exited with exit code 1 Started lustre-OST0000 oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1736264018.65 total 4.95 last 0.00) total: 100 open/close in 5.53 seconds: 18.09 ops/second umount lustre on /mnt/lustre..... Stopping client oleg206-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg206-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg206-server stop mds service on oleg206-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg206-server LNET ready to unload unloading modules on: 'oleg206-server' oleg206-server: oleg206-server.virtnet: executing unload_modules_local oleg206-server: LNET ready to unload modules unloaded.