== conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 17:51:23 (1717019483) Stopping clients: oleg431-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg431-client.virtnet /mnt/lustre2 (opts:-f) oleg431-server: oleg431-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg431-server' oleg431-server: oleg431-server.virtnet: executing load_modules_local oleg431-server: Loading modules from /home/green/git/lustre-release/lustre oleg431-server: detected 4 online CPUs by sysfs oleg431-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg431-server' oleg431-server: oleg431-server.virtnet: executing load_modules_local oleg431-server: Loading modules from /home/green/git/lustre-release/lustre oleg431-server: detected 4 online CPUs by sysfs oleg431-server: Force libcfs to create 2 CPU partitions oleg431-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory fail_loc=0x00001e0 start mds service on oleg431-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg431-server: oleg431-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg431-client: oleg431-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg431-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg431-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg431-server: oleg431-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg431-client: oleg431-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg431-client.virtnet: -o user_xattr,flock oleg431-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 2009 (time 1717019539.62 total 10.00 last 200.80) - open/close 3812 (time 1717019549.63 total 20.01 last 180.25) - open/close 5457 (time 1717019559.63 total 30.01 last 164.43) - open/close 7039 (time 1717019569.64 total 40.02 last 158.12) - open/close 8359 (time 1717019579.64 total 50.02 last 132.00) - open/close 9985 (time 1717019589.64 total 60.02 last 162.53) total: 10000 open/close in 60.09 seconds: 166.43 ops/second - unlinked 0 (time 1717019591 ; total 0 ; last 0) total: 10000 unlinks in 34 seconds: 294.117645 unlinks/second - open/close 1580 (time 1717019637.01 total 10.00 last 158.00) - open/close 3178 (time 1717019647.01 total 20.01 last 159.71) - open/close 4864 (time 1717019657.02 total 30.01 last 168.55) - open/close 6456 (time 1717019667.02 total 40.01 last 159.10) - open/close 7849 (time 1717019677.02 total 50.02 last 139.26) - open/close 9441 (time 1717019687.03 total 60.02 last 159.17) total: 10000 open/close in 63.50 seconds: 157.47 ops/second - unlinked 0 (time 1717019691 ; total 0 ; last 0) total: 10000 unlinks in 42 seconds: 238.095245 unlinks/second - open/close 1295 (time 1717019744.86 total 10.00 last 129.45) - open/close 2656 (time 1717019754.86 total 20.00 last 136.09) - open/close 3954 (time 1717019764.86 total 30.01 last 129.73) - open/close 5226 (time 1717019774.88 total 40.02 last 127.05) - open/close 6574 (time 1717019784.88 total 50.02 last 134.77) - open/close 7820 (time 1717019794.88 total 60.03 last 124.53) - open/close 9205 (time 1717019804.89 total 70.04 last 138.40) total: 10000 open/close in 75.95 seconds: 131.67 ops/second - unlinked 0 (time 1717019812 ; total 0 ; last 0) total: 10000 unlinks in 43 seconds: 232.558136 unlinks/second - open/close 1625 (time 1717019867.15 total 10.01 last 162.41) - open/close 3150 (time 1717019877.15 total 20.01 last 152.41) - open/close 4631 (time 1717019887.16 total 30.02 last 148.02) - open/close 6084 (time 1717019897.16 total 40.02 last 145.21) - open/close 7426 (time 1717019907.17 total 50.03 last 134.15) - open/close 8711 (time 1717019917.17 total 60.03 last 128.48) total: 10000 open/close in 70.02 seconds: 142.81 ops/second - unlinked 0 (time 1717019928 ; total 0 ; last 0) total: 10000 unlinks in 42 seconds: 238.095245 unlinks/second - open/close 1372 (time 1717019981.91 total 10.01 last 137.13) - open/close 2755 (time 1717019991.91 total 20.01 last 138.29) - open/close 4400 (time 1717020001.92 total 30.01 last 164.41) - open/close 5855 (time 1717020011.93 total 40.02 last 145.42) - open/close 7332 (time 1717020021.94 total 50.03 last 147.55) - open/close 8585 (time 1717020031.94 total 60.03 last 125.29) - open/close 9986 (time 1717020041.94 total 70.03 last 140.06) total: 10000 open/close in 70.09 seconds: 142.66 ops/second - unlinked 0 (time 1717020043 ; total 0 ; last 0) total: 10000 unlinks in 44 seconds: 227.272720 unlinks/second - open/close 1311 (time 1717020099.00 total 10.01 last 130.99) - open/close 2735 (time 1717020109.01 total 20.02 last 142.31) - open/close 4032 (time 1717020119.01 total 30.02 last 129.65) - open/close 5304 (time 1717020129.02 total 40.03 last 127.09) - open/close 6721 (time 1717020139.02 total 50.03 last 141.65) - open/close 8213 (time 1717020149.03 total 60.04 last 149.12) - open/close 9643 (time 1717020159.03 total 70.04 last 142.92) total: 10000 open/close in 72.25 seconds: 138.41 ops/second - unlinked 0 (time 1717020162 ; total 0 ; last 0) total: 10000 unlinks in 40 seconds: 250.000000 unlinks/second - open/close 1517 (time 1717020213.70 total 10.00 last 151.66) - open/close 2928 (time 1717020223.70 total 20.00 last 141.08) - open/close 4401 (time 1717020233.71 total 30.01 last 147.21) - open/close 5888 (time 1717020243.71 total 40.01 last 148.64) - open/close 7652 (time 1717020253.71 total 50.02 last 176.32) - open/close 9100 (time 1717020263.71 total 60.02 last 144.80) total: 10000 open/close in 65.79 seconds: 151.99 ops/second - unlinked 0 (time 1717020271 ; total 0 ; last 0) total: 10000 unlinks in 28 seconds: 357.142853 unlinks/second - open/close 1731 (time 1717020310.43 total 10.01 last 173.01) - open/close 3307 (time 1717020320.44 total 20.01 last 157.49) - open/close 4819 (time 1717020330.45 total 30.02 last 151.07) - open/close 6280 (time 1717020340.45 total 40.02 last 146.10) - open/close 7640 (time 1717020350.45 total 50.03 last 135.94) - open/close 8971 (time 1717020360.46 total 60.03 last 133.03) total: 10000 open/close in 67.59 seconds: 147.95 ops/second - unlinked 0 (time 1717020369 ; total 0 ; last 0) total: 10000 unlinks in 39 seconds: 256.410248 unlinks/second - open/close 1504 (time 1717020420.40 total 10.00 last 150.34) - open/close 3258 (time 1717020430.41 total 20.01 last 175.32) - open/close 4824 (time 1717020440.42 total 30.02 last 156.44) - open/close 6597 (time 1717020450.42 total 40.03 last 177.15) - open/close 8137 (time 1717020460.43 total 50.04 last 153.84) - open/close 9588 (time 1717020470.44 total 60.04 last 145.08) total: 10000 open/close in 63.37 seconds: 157.80 ops/second - unlinked 0 (time 1717020475 ; total 0 ; last 0) total: 10000 unlinks in 44 seconds: 227.272720 unlinks/second - open/close 1329 (time 1717020531.59 total 10.00 last 132.84) - open/close 2692 (time 1717020541.59 total 20.01 last 136.28) - open/close 3957 (time 1717020551.59 total 30.01 last 126.41) - open/close 5155 (time 1717020561.60 total 40.02 last 119.76) - open/close 6447 (time 1717020571.60 total 50.02 last 129.12) - open/close 8080 (time 1717020581.61 total 60.03 last 163.15) - open/close 9439 (time 1717020591.62 total 70.04 last 135.83) total: 10000 open/close in 74.19 seconds: 134.79 ops/second - unlinked 0 (time 1717020597 ; total 0 ; last 0) total: 10000 unlinks in 44 seconds: 227.272720 unlinks/second - open/close 1317 (time 1717020652.93 total 10.02 last 131.45) - open/close 2766 (time 1717020662.94 total 20.03 last 144.81) - open/close 4202 (time 1717020672.94 total 30.03 last 143.58) - open/close 5509 (time 1717020682.94 total 40.03 last 130.65) - open/close 6927 (time 1717020692.94 total 50.03 last 141.80) - open/close 8315 (time 1717020702.95 total 60.04 last 138.70) - open/close 9661 (time 1717020712.96 total 70.05 last 134.47) total: 10000 open/close in 72.70 seconds: 137.55 ops/second - unlinked 0 (time 1717020717 ; total 0 ; last 0) total: 10000 unlinks in 46 seconds: 217.391312 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5760 375808 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 3072 377856 1% /mnt/lustre[OST:0] filesystem_summary: 382976 3072 377856 1% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.300628 s, 3.5 MB/s fail_loc=0x80000204 Failing ost1 on oleg431-server Stopping /mnt/lustre-ost1 (opts:) on oleg431-server reboot facets: ost1 Failover ost1 to oleg431-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg431-server: oleg431-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg431-client: oleg431-server: ssh exited with exit code 1 Started lustre-OST0000 oleg431-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1717020794.32 total 2.19 last 0.00) total: 100 open/close in 3.67 seconds: 27.21 ops/second umount lustre on /mnt/lustre..... Stopping client oleg431-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg431-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg431-server stop mds service on oleg431-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg431-server LNET ready to unload unloading modules on: 'oleg431-server' oleg431-server: oleg431-server.virtnet: executing unload_modules_local oleg431-server: LNET ready to unload modules unloaded.