== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 18:24:07 (1718317447) start mds service on oleg418-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg418-server' oleg418-server: oleg418-server.virtnet: executing load_modules_local oleg418-server: Loading modules from /home/green/git/lustre-release/lustre oleg418-server: detected 4 online CPUs by sysfs oleg418-server: Force libcfs to create 2 CPU partitions oleg418-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg418-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg418-server: oleg418-server.virtnet: executing set_default_debug -1 all pdsh@oleg418-client: oleg418-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg418-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg418-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg418-server: oleg418-server.virtnet: executing set_default_debug -1 all pdsh@oleg418-client: oleg418-server: ssh exited with exit code 1 Started lustre-OST0000 oleg418-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg418-client.virtnet: -o user_xattr,flock oleg418-server@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 269 (time 1718317511.06 total 10.23 last 26.30) - open/close 419 (time 1718317521.12 total 20.29 last 14.91) - open/close 634 (time 1718317531.68 total 30.86 last 20.35) - open/close 822 (time 1718317541.93 total 41.10 last 18.36) - open/close 1037 (time 1718317552.32 total 51.49 last 20.69) - open/close 1209 (time 1718317562.81 total 61.98 last 16.40) - open/close 1432 (time 1718317573.54 total 72.71 last 20.78) - open/close 1702 (time 1718317584.47 total 83.64 last 24.71) - open/close 1842 (time 1718317595.39 total 94.56 last 12.82) - open/close 2085 (time 1718317605.40 total 104.57 last 24.28) - open/close 2261 (time 1718317615.41 total 114.58 last 17.58) - open/close 2408 (time 1718317625.49 total 124.67 last 14.58) - open/close 2587 (time 1718317635.50 total 134.67 last 17.89) - open/close 2743 (time 1718317645.50 total 144.67 last 15.60) - open/close 3055 (time 1718317655.94 total 155.11 last 29.88) - open/close 3410 (time 1718317665.98 total 165.15 last 35.37) - open/close 3640 (time 1718317676.55 total 175.72 last 21.77) - open/close 3852 (time 1718317686.74 total 185.91 last 20.80) - open/close 4109 (time 1718317696.74 total 195.91 last 25.70) - open/close 4713 (time 1718317707.02 total 206.19 last 58.76) - open/close 5367 (time 1718317717.25 total 216.42 last 63.93) - open/close 6155 (time 1718317727.77 total 226.94 last 74.86) - open/close 7061 (time 1718317738.62 total 237.79 last 83.54) - open/close 7909 (time 1718317749.29 total 248.46 last 79.49) - open/close 8780 (time 1718317759.99 total 259.16 last 81.39) - open/close 9591 (time 1718317770.57 total 269.74 last 76.64) - open/close 10000 (time 1718317776.38 total 275.55 last 70.40) - open/close 10693 (time 1718317786.72 total 285.89 last 66.99) - open/close 11278 (time 1718317796.94 total 296.11 last 57.28) - open/close 11956 (time 1718317807.23 total 306.41 last 65.83) - open/close 12507 (time 1718317817.46 total 316.64 last 53.87) - open/close 13164 (time 1718317828.06 total 327.23 last 62.01) - open/close 13643 (time 1718317838.11 total 337.28 last 47.64) - open/close 16960 (time 1718317848.11 total 347.29 last 331.69) - open/close 20000 (time 1718317853.79 total 352.96 last 535.68) - open/close 24662 (time 1718317863.79 total 362.96 last 466.17) - open/close 28388 (time 1718317873.79 total 372.96 last 372.59) - open/close 30000 (time 1718317877.72 total 376.89 last 409.94) - open/close 33360 (time 1718317887.72 total 386.89 last 335.98) - open/close 36593 (time 1718317897.72 total 396.90 last 323.26) - open/close 40000 (time 1718317907.05 total 406.23 last 365.16) - open/close 43855 (time 1718317917.05 total 416.23 last 385.46) - open/close 48472 (time 1718317927.06 total 426.23 last 461.56) total: 50000 open/close in 430.96 seconds: 116.02 ops/second - unlinked 0 (time 1718317932 ; total 0 ; last 0) - unlinked 10000 (time 1718317946 ; total 14 ; last 14) - unlinked 20000 (time 1718317958 ; total 26 ; last 12) - unlinked 30000 (time 1718317970 ; total 38 ; last 12) - unlinked 40000 (time 1718317982 ; total 50 ; last 12) total: 50000 unlinks in 61 seconds: 819.672119 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg418-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg418-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg418-server stop mds service on oleg418-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg418-server unloading modules on: 'oleg418-server' oleg418-server: oleg418-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg418-client: oleg418-client: ssh exited with exit code 2 pdsh@oleg418-client: oleg418-server: ssh exited with exit code 2 pdsh@oleg418-client: oleg418-client: ssh exited with exit code 2 pdsh@oleg418-client: oleg418-server: ssh exited with exit code 2