== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 16:58:27 (1746133107) start mds service on oleg420-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing load_modules_local oleg420-server: Loading modules from /home/green/git/lustre-release/lustre oleg420-server: detected 4 online CPUs by sysfs oleg420-server: MODOPTS_LIBCFS= oleg420-server: Force libcfs to create 2 CPU partitions oleg420-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg420-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg420-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg420-server: oleg420-server.virtnet: executing set_default_debug -1 all pdsh@oleg420-client: oleg420-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg420-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg420-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg420-server: oleg420-server.virtnet: executing set_default_debug -1 all pdsh@oleg420-client: oleg420-server: ssh exited with exit code 1 Started lustre-OST0000 oleg420-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg420-client.virtnet: -o user_xattr,flock 192.168.204.120@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 204 (time 1746133151.69 total 12.21 last 16.71) - open/close 450 (time 1746133162.11 total 22.63 last 23.60) - open/close 678 (time 1746133172.59 total 33.11 last 21.76) - open/close 786 (time 1746133182.99 total 43.51 last 10.39) - open/close 883 (time 1746133193.36 total 53.88 last 9.35) - open/close 1059 (time 1746133204.13 total 64.64 last 16.36) - open/close 1164 (time 1746133214.82 total 75.34 last 9.82) - open/close 1324 (time 1746133225.70 total 86.22 last 14.71) - open/close 1524 (time 1746133236.29 total 96.81 last 18.87) - open/close 1754 (time 1746133247.02 total 107.54 last 21.45) - open/close 2049 (time 1746133257.02 total 117.54 last 29.50) - open/close 2228 (time 1746133267.03 total 127.55 last 17.88) - open/close 2520 (time 1746133277.16 total 137.68 last 28.82) - open/close 2770 (time 1746133287.85 total 148.37 last 23.39) - open/close 3056 (time 1746133298.69 total 159.21 last 26.39) - open/close 3318 (time 1746133310.11 total 170.63 last 22.93) - open/close 3501 (time 1746133321.03 total 181.54 last 16.77) - open/close 3763 (time 1746133331.49 total 192.01 last 25.03) - open/close 4082 (time 1746133341.50 total 202.02 last 31.87) - open/close 4310 (time 1746133351.52 total 212.04 last 22.76) - open/close 4554 (time 1746133362.32 total 222.84 last 22.60) - open/close 4852 (time 1746133373.23 total 233.75 last 27.31) - open/close 5035 (time 1746133383.73 total 244.25 last 17.42) - open/close 5203 (time 1746133393.87 total 254.38 last 16.57) - open/close 5401 (time 1746133404.04 total 264.56 last 19.45) - open/close 5700 (time 1746133414.05 total 274.57 last 29.89) - open/close 5940 (time 1746133424.64 total 285.16 last 22.66) - open/close 6176 (time 1746133434.90 total 295.42 last 22.99) - open/close 6405 (time 1746133444.90 total 305.42 last 22.90) - open/close 6649 (time 1746133455.13 total 315.64 last 23.87) - open/close 6772 (time 1746133467.38 total 327.90 last 10.03) - open/close 7062 (time 1746133478.22 total 338.74 last 26.77) - open/close 7328 (time 1746133488.88 total 349.40 last 24.94) - open/close 7580 (time 1746133499.41 total 359.93 last 23.95) - open/close 7864 (time 1746133510.27 total 370.79 last 26.14) - open/close 8054 (time 1746133520.27 total 380.79 last 19.00) - open/close 8223 (time 1746133530.86 total 391.38 last 15.96) - open/close 8443 (time 1746133540.86 total 401.38 last 22.00) - open/close 8614 (time 1746133551.35 total 411.87 last 16.30) - open/close 8790 (time 1746133561.71 total 422.22 last 17.00) - open/close 9002 (time 1746133571.97 total 432.49 last 20.65) - open/close 9218 (time 1746133581.98 total 442.50 last 21.59) - open/close 9491 (time 1746133591.99 total 452.51 last 27.27) - open/close 9777 (time 1746133601.99 total 462.51 last 28.59) - open/close 11353 (time 1746133611.99 total 472.51 last 157.58) - open/close 12664 (time 1746133621.99 total 482.51 last 131.09) - open/close 14564 (time 1746133632.00 total 492.51 last 189.95) - open/close 16469 (time 1746133642.00 total 502.52 last 190.42) - open/close 18459 (time 1746133652.00 total 512.52 last 198.95) - open/close 19903 (time 1746133662.01 total 522.53 last 144.34) - open/close 21173 (time 1746133672.01 total 532.53 last 126.99) - open/close 22528 (time 1746133682.01 total 542.53 last 135.46) - open/close 24379 (time 1746133692.01 total 552.53 last 185.10) - open/close 25781 (time 1746133702.02 total 562.54 last 140.11) - open/close 27340 (time 1746133712.02 total 572.54 last 155.89) - open/close 29036 (time 1746133722.02 total 582.54 last 169.56) - open/close 30000 (time 1746133727.02 total 587.53 last 192.96) - open/close 32439 (time 1746133737.02 total 597.54 last 243.84) - open/close 34096 (time 1746133747.03 total 607.55 last 165.56) - open/close 36227 (time 1746133757.03 total 617.55 last 213.02) - open/close 38031 (time 1746133767.03 total 627.55 last 180.35) - open/close 40000 (time 1746133776.04 total 636.55 last 218.75) - open/close 41998 (time 1746133786.04 total 646.55 last 199.78) - open/close 44217 (time 1746133796.04 total 656.56 last 221.89) - open/close 46591 (time 1746133806.04 total 666.56 last 237.37) - open/close 49359 (time 1746133816.04 total 676.56 last 276.71) total: 50000 open/close in 679.30 seconds: 73.61 ops/second - unlinked 0 (time 1746133819 ; total 0 ; last 0) - unlinked 10000 (time 1746133848 ; total 29 ; last 29) - unlinked 20000 (time 1746133875 ; total 56 ; last 27) - unlinked 30000 (time 1746133910 ; total 91 ; last 35) - unlinked 40000 (time 1746133947 ; total 128 ; last 37) total: 50000 unlinks in 148 seconds: 337.837830 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg420-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg420-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg420-server stop mds service on oleg420-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg420-server unloading modules via unload_modules_local on: 'oleg420-server' oleg420-server: oleg420-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg420-client: oleg420-client: ssh exited with exit code 2 pdsh@oleg420-client: oleg420-server: ssh exited with exit code 2 pdsh@oleg420-client: oleg420-client: ssh exited with exit code 2 pdsh@oleg420-client: oleg420-server: ssh exited with exit code 2