== conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 09:59:08 (1747403948) Stopping clients: oleg406-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg406-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg406-client: oleg406-server: ssh exited with exit code 2 oleg406-server: oleg406-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg406-server' oleg406-server: oleg406-server.virtnet: executing load_modules_local oleg406-server: Loading modules from /home/green/git/lustre-release/lustre oleg406-server: detected 4 online CPUs by sysfs oleg406-server: Force libcfs to create 2 CPU partitions oleg406-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg406-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format mds2: /dev/mapper/mds2_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg406-server' oleg406-server: oleg406-server.virtnet: executing load_modules_local oleg406-server: Loading modules from /home/green/git/lustre-release/lustre oleg406-server: detected 4 online CPUs by sysfs oleg406-server: Force libcfs to create 2 CPU partitions oleg406-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory fail_loc=0x00001e0 start mds service on oleg406-server Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 start mds service on oleg406-server Starting mds2: -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds2_flakey Started lustre-MDT0001 oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg406-server Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg406-client.virtnet: -o user_xattr,flock oleg406-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 49667 inodes available. Want 100500. rc=0 - open/close 3515 (time 1747404008.42 total 10.00 last 351.38) - open/close 6464 (time 1747404018.42 total 20.01 last 294.85) - open/close 9189 (time 1747404028.43 total 30.01 last 272.42) total: 10000 open/close in 32.08 seconds: 311.74 ops/second - unlinked 0 (time 1747404031 ; total 0 ; last 0) total: 10000 unlinks in 19 seconds: 526.315796 unlinks/second - open/close 3058 (time 1747404061.86 total 10.00 last 305.76) - open/close 6445 (time 1747404071.86 total 20.00 last 338.68) - open/close 9600 (time 1747404081.86 total 30.00 last 315.44) total: 10000 open/close in 31.17 seconds: 320.86 ops/second - unlinked 0 (time 1747404084 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 3582 (time 1747404107.86 total 10.00 last 358.18) - open/close 7095 (time 1747404117.86 total 20.00 last 351.29) total: 10000 open/close in 28.45 seconds: 351.47 ops/second - unlinked 0 (time 1747404127 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 3413 (time 1747404152.11 total 10.00 last 341.18) - open/close 6816 (time 1747404162.11 total 20.00 last 340.27) total: 10000 open/close in 29.33 seconds: 340.98 ops/second - unlinked 0 (time 1747404172 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 3546 (time 1747404196.34 total 10.00 last 354.58) - open/close 7105 (time 1747404206.34 total 20.00 last 355.81) total: 10000 open/close in 27.57 seconds: 362.71 ops/second - unlinked 0 (time 1747404215 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 5543 (time 1747404238.11 total 10.00 last 554.24) total: 10000 open/close in 19.22 seconds: 520.38 ops/second - unlinked 0 (time 1747404248 ; total 0 ; last 0) total: 10000 unlinks in 8 seconds: 1250.000000 unlinks/second - open/close 5967 (time 1747404266.91 total 10.00 last 596.66) total: 10000 open/close in 18.54 seconds: 539.24 ops/second - unlinked 0 (time 1747404276 ; total 0 ; last 0) total: 10000 unlinks in 10 seconds: 1000.000000 unlinks/second - open/close 4519 (time 1747404296.93 total 10.00 last 451.88) - open/close 9033 (time 1747404306.94 total 20.00 last 451.25) total: 10000 open/close in 23.27 seconds: 429.67 ops/second - unlinked 0 (time 1747404311 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 3796 (time 1747404333.20 total 10.00 last 379.50) - open/close 7786 (time 1747404343.20 total 20.01 last 398.86) total: 10000 open/close in 26.20 seconds: 381.67 ops/second - unlinked 0 (time 1747404350 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second - open/close 3495 (time 1747404376.43 total 10.00 last 349.49) - open/close 6691 (time 1747404386.43 total 20.00 last 319.55) total: 10000 open/close in 28.74 seconds: 347.95 ops/second - unlinked 0 (time 1747404395 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 3776 (time 1747404418.91 total 10.00 last 377.58) - open/close 7323 (time 1747404428.92 total 20.00 last 354.58) total: 10000 open/close in 27.85 seconds: 359.11 ops/second - unlinked 0 (time 1747404437 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 95248 4200 82392 5% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 95248 1504 85088 2% /mnt/lustre[MDT:1] lustre-OST0000_UUID 142216 1644 126572 2% /mnt/lustre[OST:0] filesystem_summary: 142216 1644 126572 2% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118233 s, 8.9 MB/s fail_loc=0x80000204 Failing ost1 on oleg406-server Stopping /mnt/lustre-ost1 (opts:) on oleg406-server reboot facets: ost1 Failover ost1 to oleg406-server mount facets: ost1 Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Started lustre-OST0000 oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1747404478.75 total 2.93 last 0.00) total: 100 open/close in 3.34 seconds: 29.91 ops/second umount lustre on /mnt/lustre..... Stopping client oleg406-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg406-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg406-server stop mds service on oleg406-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg406-server stop mds service on oleg406-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg406-server LNET ready to unload unloading modules on: 'oleg406-server' oleg406-server: oleg406-server.virtnet: executing unload_modules_local oleg406-server: LNET ready to unload modules unloaded.