== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 13:54:09 (1750182849) start mds service on oleg127-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg127-server' oleg127-server: oleg127-server.virtnet: executing load_modules_local oleg127-server: Loading modules from /home/green/git/lustre-release/lustre oleg127-server: detected 4 online CPUs by sysfs oleg127-server: MODOPTS_LIBCFS= oleg127-server: Force libcfs to create 2 CPU partitions oleg127-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg127-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg127-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg127-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg127-server: oleg127-server.virtnet: executing set_default_debug -1 all pdsh@oleg127-client: oleg127-server: ssh exited with exit code 1 Started lustre-OST0000 oleg127-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg127-client.virtnet: -o user_xattr,flock 192.168.201.127@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 246 (time 1750182887.56 total 10.78 last 22.82) - open/close 466 (time 1750182898.02 total 21.24 last 21.03) - open/close 875 (time 1750182908.02 total 31.24 last 40.90) - open/close 1135 (time 1750182918.53 total 41.74 last 24.75) - open/close 1408 (time 1750182929.52 total 52.74 last 24.83) - open/close 1665 (time 1750182939.60 total 62.82 last 25.50) - open/close 1971 (time 1750182950.06 total 73.28 last 29.26) - open/close 2298 (time 1750182960.52 total 83.74 last 31.26) - open/close 2612 (time 1750182971.15 total 94.37 last 29.52) - open/close 2921 (time 1750182981.56 total 104.78 last 29.69) - open/close 3227 (time 1750182992.03 total 115.25 last 29.23) - open/close 3412 (time 1750183002.25 total 125.46 last 18.11) - open/close 3599 (time 1750183012.60 total 135.82 last 18.06) - open/close 3838 (time 1750183023.03 total 146.24 last 22.92) - open/close 4042 (time 1750183033.21 total 156.43 last 20.02) - open/close 4226 (time 1750183043.43 total 166.64 last 18.02) - open/close 4543 (time 1750183055.02 total 178.24 last 27.33) - open/close 4822 (time 1750183065.27 total 188.49 last 27.22) - open/close 5092 (time 1750183075.39 total 198.61 last 26.69) - open/close 5315 (time 1750183085.72 total 208.93 last 21.60) - open/close 5627 (time 1750183096.71 total 219.93 last 28.38) - open/close 5882 (time 1750183108.98 total 232.19 last 20.79) - open/close 6201 (time 1750183119.42 total 242.63 last 30.55) - open/close 6500 (time 1750183130.42 total 253.64 last 27.17) - open/close 6817 (time 1750183141.16 total 264.37 last 29.54) - open/close 7073 (time 1750183151.59 total 274.81 last 24.53) - open/close 7311 (time 1750183162.19 total 285.41 last 22.45) - open/close 7660 (time 1750183173.15 total 296.36 last 31.86) - open/close 7988 (time 1750183183.91 total 307.12 last 30.48) - open/close 8257 (time 1750183193.98 total 317.20 last 26.70) - open/close 8510 (time 1750183204.92 total 328.13 last 23.14) - open/close 8820 (time 1750183215.59 total 338.81 last 29.04) - open/close 9070 (time 1750183225.60 total 348.82 last 24.98) - open/close 9376 (time 1750183236.09 total 359.31 last 29.16) - open/close 9643 (time 1750183246.37 total 369.59 last 25.96) - open/close 9878 (time 1750183257.17 total 380.39 last 21.77) - open/close 10000 (time 1750183260.17 total 383.39 last 40.68) - open/close 10201 (time 1750183271.97 total 395.19 last 17.03) - open/close 10553 (time 1750183282.13 total 405.35 last 34.65) - open/close 10806 (time 1750183292.40 total 415.61 last 24.64) - open/close 11908 (time 1750183302.40 total 425.62 last 110.15) - open/close 13270 (time 1750183312.41 total 435.62 last 136.15) - open/close 14628 (time 1750183322.42 total 445.63 last 135.66) - open/close 16046 (time 1750183332.43 total 455.65 last 141.63) - open/close 17585 (time 1750183342.43 total 465.65 last 153.81) - open/close 19083 (time 1750183352.44 total 475.66 last 149.70) - open/close 20000 (time 1750183357.48 total 480.69 last 182.09) - open/close 21594 (time 1750183367.48 total 490.70 last 159.31) - open/close 23170 (time 1750183377.48 total 500.70 last 157.60) - open/close 24637 (time 1750183387.49 total 510.70 last 146.65) - open/close 26156 (time 1750183397.49 total 520.70 last 151.89) - open/close 27686 (time 1750183407.49 total 530.70 last 152.97) - open/close 29206 (time 1750183417.49 total 540.71 last 151.96) - open/close 30000 (time 1750183422.69 total 545.91 last 152.68) - open/close 31632 (time 1750183432.70 total 555.91 last 163.08) - open/close 33290 (time 1750183442.70 total 565.92 last 165.75) - open/close 34689 (time 1750183452.70 total 575.92 last 139.86) - open/close 36094 (time 1750183462.71 total 585.92 last 140.45) - open/close 37684 (time 1750183472.71 total 595.93 last 158.93) - open/close 39536 (time 1750183482.72 total 605.93 last 185.11) - open/close 40000 (time 1750183485.23 total 608.45 last 184.61) - open/close 41880 (time 1750183495.23 total 618.45 last 187.97) - open/close 43782 (time 1750183505.24 total 628.45 last 190.11) - open/close 45267 (time 1750183515.24 total 638.46 last 148.41) - open/close 46851 (time 1750183525.24 total 648.46 last 158.40) - open/close 48595 (time 1750183535.25 total 658.46 last 174.30) total: 50000 open/close in 665.86 seconds: 75.09 ops/second - unlinked 0 (time 1750183544 ; total 0 ; last 0) - unlinked 10000 (time 1750183587 ; total 43 ; last 43) - unlinked 20000 (time 1750183614 ; total 70 ; last 27) - unlinked 30000 (time 1750183641 ; total 97 ; last 27) - unlinked 40000 (time 1750183681 ; total 137 ; last 40) total: 50000 unlinks in 179 seconds: 279.329620 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg127-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg127-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg127-server stop mds service on oleg127-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg127-server unloading modules via unload_modules_local on: 'oleg127-server' oleg127-server: oleg127-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg127-client: oleg127-client: ssh exited with exit code 2 pdsh@oleg127-client: oleg127-server: ssh exited with exit code 2 pdsh@oleg127-client: oleg127-client: ssh exited with exit code 2 pdsh@oleg127-client: oleg127-server: ssh exited with exit code 2