== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 16:16:11 (1746130571) start mds service on oleg321-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: MODOPTS_LIBCFS= oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg321-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg321-server: oleg321-server.virtnet: executing set_default_debug -1 all pdsh@oleg321-client: oleg321-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg321-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg321-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg321-server: oleg321-server.virtnet: executing set_default_debug -1 all pdsh@oleg321-client: oleg321-server: ssh exited with exit code 1 Started lustre-OST0000 oleg321-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg321-client.virtnet: -o user_xattr,flock 192.168.203.121@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 164 (time 1746130612.76 total 13.55 last 12.10) - open/close 335 (time 1746130623.48 total 24.28 last 15.95) - open/close 547 (time 1746130633.48 total 34.28 last 21.20) - open/close 813 (time 1746130644.06 total 44.85 last 25.16) - open/close 1048 (time 1746130654.12 total 54.91 last 23.36) - open/close 1360 (time 1746130664.98 total 65.78 last 28.71) - open/close 1589 (time 1746130674.99 total 75.78 last 22.89) - open/close 1847 (time 1746130684.99 total 85.79 last 25.80) - open/close 2010 (time 1746130695.08 total 95.87 last 16.16) - open/close 2239 (time 1746130705.35 total 106.15 last 22.29) - open/close 2516 (time 1746130715.88 total 116.67 last 26.31) - open/close 2583 (time 1746130726.82 total 127.62 last 6.12) - open/close 2709 (time 1746130737.52 total 138.31 last 11.78) - open/close 2820 (time 1746130748.27 total 149.06 last 10.32) - open/close 2898 (time 1746130760.65 total 161.45 last 6.30) - open/close 3125 (time 1746130771.19 total 171.98 last 21.55) - open/close 3380 (time 1746130782.08 total 182.87 last 23.41) - open/close 3569 (time 1746130792.30 total 193.09 last 18.50) - open/close 3788 (time 1746130802.88 total 203.68 last 20.69) - open/close 4070 (time 1746130813.66 total 214.45 last 26.17) - open/close 4312 (time 1746130824.14 total 224.94 last 23.08) - open/close 4554 (time 1746130834.56 total 235.35 last 23.24) - open/close 4832 (time 1746130845.26 total 246.05 last 25.97) - open/close 5130 (time 1746130856.20 total 257.00 last 27.24) - open/close 5424 (time 1746130866.54 total 267.34 last 28.42) - open/close 5608 (time 1746130876.55 total 277.35 last 18.39) - open/close 5786 (time 1746130886.55 total 287.35 last 17.79) - open/close 6014 (time 1746130896.86 total 297.65 last 22.13) - open/close 6239 (time 1746130906.87 total 307.67 last 22.47) - open/close 6444 (time 1746130917.30 total 318.10 last 19.65) - open/close 6640 (time 1746130927.34 total 328.14 last 19.52) - open/close 6782 (time 1746130937.95 total 338.74 last 13.39) - open/close 6929 (time 1746130948.93 total 349.72 last 13.39) - open/close 6984 (time 1746130959.71 total 360.51 last 5.10) - open/close 7229 (time 1746130969.72 total 370.51 last 24.48) - open/close 7297 (time 1746130980.71 total 381.51 last 6.19) - open/close 7389 (time 1746130991.99 total 392.78 last 8.16) - open/close 7634 (time 1746131001.99 total 402.78 last 24.49) - open/close 7919 (time 1746131012.82 total 413.61 last 26.31) - open/close 8100 (time 1746131023.03 total 423.82 last 17.73) - open/close 8325 (time 1746131033.47 total 434.26 last 21.54) - open/close 8507 (time 1746131044.42 total 445.21 last 16.63) - open/close 8724 (time 1746131054.90 total 455.70 last 20.69) - open/close 8973 (time 1746131065.21 total 466.01 last 24.15) - open/close 9177 (time 1746131076.35 total 477.14 last 18.32) - open/close 10000 (time 1746131084.93 total 485.73 last 95.88) - open/close 11072 (time 1746131094.93 total 495.73 last 107.19) - open/close 12378 (time 1746131104.94 total 505.73 last 130.57) - open/close 13495 (time 1746131114.94 total 515.74 last 111.65) - open/close 14556 (time 1746131124.95 total 525.74 last 106.02) - open/close 15681 (time 1746131134.95 total 535.74 last 112.49) - open/close 16822 (time 1746131144.96 total 545.75 last 114.00) - open/close 17802 (time 1746131154.98 total 555.77 last 97.84) - open/close 18745 (time 1746131164.98 total 565.78 last 94.24) - open/close 19877 (time 1746131174.99 total 575.78 last 113.17) - open/close 21071 (time 1746131184.99 total 585.79 last 119.33) - open/close 21988 (time 1746131194.99 total 595.79 last 91.67) - open/close 22939 (time 1746131205.00 total 605.80 last 95.03) - open/close 24044 (time 1746131215.01 total 615.81 last 110.41) - open/close 25086 (time 1746131225.02 total 625.81 last 104.11) - open/close 25998 (time 1746131235.02 total 635.82 last 91.17) - open/close 27031 (time 1746131245.03 total 645.82 last 103.28) - open/close 28237 (time 1746131255.03 total 655.82 last 120.53) - open/close 29119 (time 1746131265.03 total 665.83 last 88.17) - open/close 30000 (time 1746131274.19 total 674.98 last 96.24) - open/close 30973 (time 1746131284.19 total 684.99 last 97.25) - open/close 32070 (time 1746131294.20 total 694.99 last 109.64) - open/close 33019 (time 1746131304.21 total 705.01 last 94.79) - open/close 33934 (time 1746131314.22 total 715.01 last 91.42) - open/close 34801 (time 1746131324.22 total 725.02 last 86.66) - open/close 35803 (time 1746131334.23 total 735.03 last 100.13) - open/close 36674 (time 1746131344.23 total 745.03 last 87.07) - open/close 37643 (time 1746131354.24 total 755.03 last 96.90) - open/close 38574 (time 1746131364.24 total 765.04 last 93.05) - open/close 39550 (time 1746131374.24 total 775.04 last 97.56) - open/close 40000 (time 1746131378.41 total 779.20 last 108.08) - open/close 40991 (time 1746131388.42 total 789.21 last 99.03) - open/close 41869 (time 1746131398.42 total 799.22 last 87.74) - open/close 42850 (time 1746131408.43 total 809.22 last 98.03) - open/close 43819 (time 1746131418.43 total 819.23 last 96.88) - open/close 44741 (time 1746131428.44 total 829.23 last 92.15) - open/close 45689 (time 1746131438.44 total 839.24 last 94.74) - open/close 46675 (time 1746131448.45 total 849.24 last 98.54) - open/close 47736 (time 1746131458.45 total 859.25 last 106.06) - open/close 48844 (time 1746131468.45 total 869.25 last 110.78) - open/close 49922 (time 1746131478.46 total 879.26 last 107.72) total: 50000 open/close in 880.03 seconds: 56.82 ops/second - unlinked 0 (time 1746131481 ; total 0 ; last 0) - unlinked 10000 (time 1746131570 ; total 89 ; last 89) - unlinked 20000 (time 1746131633 ; total 152 ; last 63) - unlinked 30000 (time 1746131701 ; total 220 ; last 68) - unlinked 40000 (time 1746131764 ; total 283 ; last 63) total: 50000 unlinks in 345 seconds: 144.927536 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg321-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg321-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg321-server stop mds service on oleg321-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg321-server unloading modules via unload_modules_local on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg321-client: oleg321-client: ssh exited with exit code 2 pdsh@oleg321-client: oleg321-server: ssh exited with exit code 2 pdsh@oleg321-client: oleg321-client: ssh exited with exit code 2 pdsh@oleg321-client: oleg321-server: ssh exited with exit code 2