== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 20:42:47 (1746146567) start mds service on oleg414-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg414-server' oleg414-server: oleg414-server.virtnet: executing load_modules_local oleg414-server: Loading modules from /home/green/git/lustre-release/lustre oleg414-server: detected 4 online CPUs by sysfs oleg414-server: MODOPTS_LIBCFS= oleg414-server: Force libcfs to create 2 CPU partitions oleg414-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg414-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg414-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg414-server: oleg414-server.virtnet: executing set_default_debug -1 all pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg414-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg414-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg414-server: oleg414-server.virtnet: executing set_default_debug -1 all pdsh@oleg414-client: oleg414-server: ssh exited with exit code 1 Started lustre-OST0000 oleg414-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg414-client.virtnet: -o user_xattr,flock 192.168.204.114@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 224 (time 1746146605.63 total 10.88 last 20.59) - open/close 457 (time 1746146615.63 total 20.88 last 23.30) - open/close 620 (time 1746146626.08 total 31.33 last 15.60) - open/close 844 (time 1746146636.61 total 41.86 last 21.26) - open/close 1019 (time 1746146647.23 total 52.48 last 16.48) - open/close 1208 (time 1746146657.85 total 63.10 last 17.80) - open/close 1408 (time 1746146667.85 total 73.10 last 20.00) - open/close 1638 (time 1746146678.83 total 84.08 last 20.95) - open/close 1906 (time 1746146688.83 total 94.09 last 26.79) - open/close 2086 (time 1746146699.70 total 104.96 last 16.56) - open/close 2370 (time 1746146710.19 total 115.44 last 27.09) - open/close 2598 (time 1746146720.20 total 125.45 last 22.77) - open/close 2810 (time 1746146731.05 total 136.30 last 19.54) - open/close 3015 (time 1746146741.75 total 147.00 last 19.16) - open/close 3254 (time 1746146752.00 total 157.26 last 23.31) - open/close 3418 (time 1746146762.27 total 167.53 last 15.97) - open/close 3675 (time 1746146773.07 total 178.32 last 23.81) - open/close 3873 (time 1746146783.45 total 188.70 last 19.08) - open/close 4096 (time 1746146794.02 total 199.27 last 21.10) - open/close 4274 (time 1746146804.62 total 209.88 last 16.78) - open/close 4502 (time 1746146815.33 total 220.58 last 21.30) - open/close 4752 (time 1746146826.08 total 231.33 last 23.27) - open/close 5014 (time 1746146836.86 total 242.11 last 24.29) - open/close 5278 (time 1746146847.02 total 252.27 last 25.99) - open/close 5556 (time 1746146857.46 total 262.71 last 26.63) - open/close 5766 (time 1746146868.48 total 273.73 last 19.05) - open/close 5946 (time 1746146878.49 total 283.74 last 17.98) - open/close 6260 (time 1746146888.56 total 293.82 last 31.17) - open/close 6501 (time 1746146898.85 total 304.10 last 23.43) - open/close 6666 (time 1746146909.10 total 314.36 last 16.09) - open/close 6859 (time 1746146919.69 total 324.94 last 18.23) - open/close 7097 (time 1746146929.75 total 335.00 last 23.67) - open/close 7354 (time 1746146940.00 total 345.25 last 25.07) - open/close 7605 (time 1746146950.25 total 355.50 last 24.49) - open/close 7804 (time 1746146960.25 total 365.50 last 19.89) - open/close 7934 (time 1746146970.89 total 376.14 last 12.22) - open/close 8094 (time 1746146981.00 total 386.25 last 15.82) - open/close 8301 (time 1746146991.53 total 396.78 last 19.66) - open/close 8609 (time 1746147002.34 total 407.59 last 28.50) - open/close 8910 (time 1746147012.47 total 417.72 last 29.71) - open/close 10000 (time 1746147021.35 total 426.60 last 122.79) - open/close 11364 (time 1746147031.35 total 436.60 last 136.38) - open/close 12784 (time 1746147041.35 total 446.60 last 141.97) - open/close 14078 (time 1746147051.35 total 456.61 last 129.37) - open/close 15420 (time 1746147061.36 total 466.61 last 134.17) - open/close 16925 (time 1746147071.36 total 476.61 last 150.44) - open/close 18416 (time 1746147081.37 total 486.62 last 148.97) - open/close 20000 (time 1746147090.62 total 495.88 last 171.14) - open/close 21372 (time 1746147100.62 total 505.88 last 137.20) - open/close 22561 (time 1746147110.63 total 515.88 last 118.84) - open/close 23829 (time 1746147120.63 total 525.89 last 126.76) - open/close 25160 (time 1746147130.64 total 535.89 last 133.01) - open/close 26481 (time 1746147140.64 total 545.90 last 132.04) - open/close 27775 (time 1746147150.65 total 555.90 last 129.32) - open/close 29147 (time 1746147160.66 total 565.91 last 137.06) - open/close 30000 (time 1746147168.08 total 573.33 last 114.98) - open/close 31203 (time 1746147178.09 total 583.34 last 120.19) - open/close 32391 (time 1746147188.09 total 593.34 last 118.75) - open/close 33747 (time 1746147198.09 total 603.35 last 135.57) - open/close 35053 (time 1746147208.10 total 613.35 last 130.56) - open/close 36237 (time 1746147218.11 total 623.36 last 118.28) - open/close 37526 (time 1746147228.11 total 633.36 last 128.84) - open/close 38659 (time 1746147238.12 total 643.38 last 113.17) - open/close 39832 (time 1746147248.13 total 653.39 last 117.19) - open/close 41237 (time 1746147258.14 total 663.39 last 140.46) - open/close 42691 (time 1746147268.14 total 673.39 last 145.40) - open/close 44349 (time 1746147278.14 total 683.39 last 165.71) - open/close 45764 (time 1746147288.15 total 693.40 last 141.42) - open/close 47189 (time 1746147298.15 total 703.40 last 142.48) - open/close 48639 (time 1746147308.15 total 713.40 last 145.00) total: 50000 open/close in 722.80 seconds: 69.18 ops/second - unlinked 0 (time 1746147319 ; total 0 ; last 0) - unlinked 10000 (time 1746147373 ; total 54 ; last 54) - unlinked 20000 (time 1746147418 ; total 99 ; last 45) - unlinked 30000 (time 1746147461 ; total 142 ; last 43) - unlinked 40000 (time 1746147504 ; total 185 ; last 43) total: 50000 unlinks in 221 seconds: 226.244339 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg414-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg414-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg414-server stop mds service on oleg414-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg414-server unloading modules via unload_modules_local on: 'oleg414-server' oleg414-server: oleg414-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg414-client: oleg414-client: ssh exited with exit code 2 pdsh@oleg414-client: oleg414-server: ssh exited with exit code 2 pdsh@oleg414-client: oleg414-server: ssh exited with exit code 2 pdsh@oleg414-client: oleg414-client: ssh exited with exit code 2