== conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 13:12:20 (1721409140) Stopping clients: oleg109-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg109-client.virtnet /mnt/lustre2 (opts:-f) oleg109-server: oleg109-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions oleg109-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory fail_loc=0x00001e0 start mds service on oleg109-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg109-server: oleg109-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg109-client: oleg109-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg109-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg109-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg109-server: oleg109-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg109-client: oleg109-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg109-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 4204 (time 1721409182.05 total 10.00 last 420.36) - open/close 7498 (time 1721409192.05 total 20.00 last 329.38) total: 10000 open/close in 26.75 seconds: 373.86 ops/second - unlinked 0 (time 1721409200 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second - open/close 2531 (time 1721409232.22 total 10.00 last 253.03) - open/close 5004 (time 1721409242.22 total 20.00 last 247.27) - open/close 7567 (time 1721409252.22 total 30.01 last 256.25) total: 10000 open/close in 39.33 seconds: 254.26 ops/second - unlinked 0 (time 1721409262 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 3085 (time 1721409294.69 total 10.00 last 308.50) - open/close 6096 (time 1721409304.69 total 20.00 last 301.09) - open/close 8730 (time 1721409314.69 total 30.00 last 263.36) total: 10000 open/close in 35.12 seconds: 284.70 ops/second - unlinked 0 (time 1721409321 ; total 0 ; last 0) total: 10000 unlinks in 19 seconds: 526.315796 unlinks/second - open/close 2812 (time 1721409351.24 total 10.00 last 281.12) - open/close 5471 (time 1721409361.24 total 20.00 last 265.88) - open/close 8059 (time 1721409371.24 total 30.00 last 258.77) total: 10000 open/close in 36.79 seconds: 271.82 ops/second - unlinked 0 (time 1721409379 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2531 (time 1721409411.50 total 10.00 last 253.03) - open/close 5182 (time 1721409421.50 total 20.00 last 265.06) - open/close 7709 (time 1721409431.50 total 30.00 last 252.69) total: 10000 open/close in 38.85 seconds: 257.37 ops/second - unlinked 0 (time 1721409441 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2618 (time 1721409473.90 total 10.00 last 261.69) - open/close 5138 (time 1721409483.90 total 20.00 last 251.99) - open/close 7828 (time 1721409493.90 total 30.01 last 268.98) total: 10000 open/close in 38.76 seconds: 258.00 ops/second - unlinked 0 (time 1721409503 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2550 (time 1721409536.19 total 10.00 last 254.92) - open/close 5271 (time 1721409546.19 total 20.00 last 272.10) - open/close 8041 (time 1721409556.19 total 30.01 last 276.91) total: 10000 open/close in 37.87 seconds: 264.06 ops/second - unlinked 0 (time 1721409565 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second - open/close 2705 (time 1721409597.12 total 10.00 last 270.42) - open/close 5194 (time 1721409607.12 total 20.00 last 248.89) - open/close 7753 (time 1721409617.12 total 30.00 last 255.88) total: 10000 open/close in 38.92 seconds: 256.94 ops/second - unlinked 0 (time 1721409627 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2629 (time 1721409659.83 total 10.00 last 262.81) - open/close 5143 (time 1721409669.83 total 20.00 last 251.38) - open/close 7748 (time 1721409679.83 total 30.01 last 260.41) total: 10000 open/close in 39.33 seconds: 254.29 ops/second - unlinked 0 (time 1721409690 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2675 (time 1721409723.08 total 10.00 last 267.50) - open/close 5193 (time 1721409733.08 total 20.00 last 251.71) - open/close 7811 (time 1721409743.09 total 30.01 last 261.73) total: 10000 open/close in 38.44 seconds: 260.16 ops/second - unlinked 0 (time 1721409752 ; total 0 ; last 0) total: 10000 unlinks in 21 seconds: 476.190491 unlinks/second - open/close 2597 (time 1721409784.72 total 10.00 last 259.63) - open/close 5136 (time 1721409794.72 total 20.00 last 253.89) - open/close 7695 (time 1721409804.72 total 30.01 last 255.82) total: 10000 open/close in 39.04 seconds: 256.12 ops/second - unlinked 0 (time 1721409815 ; total 0 ; last 0) total: 10000 unlinks in 19 seconds: 526.315796 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5760 375808 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 3072 377856 1% /mnt/lustre[OST:0] filesystem_summary: 382976 3072 377856 1% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.227677 s, 4.6 MB/s fail_loc=0x80000204 Failing ost1 on oleg109-server Stopping /mnt/lustre-ost1 (opts:) on oleg109-server reboot facets: ost1 Failover ost1 to oleg109-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg109-server: oleg109-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg109-client: oleg109-server: ssh exited with exit code 1 Started lustre-OST0000 oleg109-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1721409863.31 total 3.01 last 0.00) total: 100 open/close in 3.81 seconds: 26.27 ops/second umount lustre on /mnt/lustre..... Stopping client oleg109-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg109-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg109-server stop mds service on oleg109-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg109-server LNET ready to unload unloading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing unload_modules_local oleg109-server: LNET ready to unload modules unloaded.