== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 00:06:17 (1746072377) start mds service on oleg234-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg234-server' oleg234-server: oleg234-server.virtnet: executing load_modules_local oleg234-server: Loading modules from /home/green/git/lustre-release/lustre oleg234-server: detected 4 online CPUs by sysfs oleg234-server: MODOPTS_LIBCFS= oleg234-server: Force libcfs to create 2 CPU partitions oleg234-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg234-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg234-server: oleg234-server.virtnet: executing set_default_debug -1 all pdsh@oleg234-client: oleg234-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg234-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg234-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg234-server: oleg234-server.virtnet: executing set_default_debug -1 all pdsh@oleg234-client: oleg234-server: ssh exited with exit code 1 Started lustre-OST0000 oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg234-client.virtnet: -o user_xattr,flock 192.168.202.134@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 310 (time 1746072416.65 total 10.70 last 28.98) - open/close 533 (time 1746072427.85 total 21.90 last 19.90) - open/close 738 (time 1746072438.74 total 32.79 last 18.83) - open/close 966 (time 1746072448.92 total 42.97 last 22.39) - open/close 1246 (time 1746072459.30 total 53.35 last 26.99) - open/close 1517 (time 1746072469.59 total 63.64 last 26.34) - open/close 1784 (time 1746072481.75 total 75.80 last 21.95) - open/close 2005 (time 1746072492.67 total 86.72 last 20.24) - open/close 2148 (time 1746072505.22 total 99.27 last 11.39) - open/close 2471 (time 1746072516.12 total 110.18 last 29.62) - open/close 2751 (time 1746072527.01 total 121.06 last 25.73) - open/close 3075 (time 1746072537.35 total 131.40 last 31.34) - open/close 3325 (time 1746072547.51 total 141.56 last 24.61) - open/close 3460 (time 1746072558.03 total 152.09 last 12.82) - open/close 3669 (time 1746072568.91 total 162.96 last 19.23) - open/close 4030 (time 1746072579.56 total 173.61 last 33.88) - open/close 4346 (time 1746072590.20 total 184.25 last 29.71) - open/close 4565 (time 1746072601.10 total 195.15 last 20.10) - open/close 4776 (time 1746072612.30 total 206.35 last 18.83) - open/close 4932 (time 1746072622.76 total 216.81 last 14.91) - open/close 5289 (time 1746072633.65 total 227.70 last 32.80) - open/close 5523 (time 1746072643.90 total 237.95 last 22.82) - open/close 5877 (time 1746072654.42 total 248.48 last 33.64) - open/close 6135 (time 1746072664.92 total 258.97 last 24.58) - open/close 6377 (time 1746072674.92 total 268.97 last 24.20) - open/close 6542 (time 1746072685.61 total 279.67 last 15.43) - open/close 8447 (time 1746072695.62 total 289.67 last 190.48) - open/close 10000 (time 1746072704.52 total 298.57 last 174.47) - open/close 12267 (time 1746072714.52 total 308.57 last 226.61) - open/close 13973 (time 1746072724.52 total 318.57 last 170.58) - open/close 15360 (time 1746072734.53 total 328.58 last 138.62) - open/close 16629 (time 1746072744.53 total 338.58 last 126.85) - open/close 17925 (time 1746072754.53 total 348.58 last 129.59) - open/close 19732 (time 1746072764.53 total 358.59 last 180.68) - open/close 21346 (time 1746072774.54 total 368.59 last 161.34) - open/close 23092 (time 1746072784.54 total 378.59 last 174.57) - open/close 24721 (time 1746072794.54 total 388.59 last 162.84) - open/close 26210 (time 1746072804.55 total 398.60 last 148.83) - open/close 27775 (time 1746072814.55 total 408.60 last 156.46) - open/close 29409 (time 1746072824.55 total 418.60 last 163.37) - open/close 30000 (time 1746072827.32 total 421.37 last 213.51) - open/close 31523 (time 1746072837.32 total 431.38 last 152.24) - open/close 33135 (time 1746072847.33 total 441.38 last 161.17) - open/close 34876 (time 1746072857.33 total 451.38 last 174.04) - open/close 37360 (time 1746072867.33 total 461.39 last 248.29) - open/close 39416 (time 1746072877.33 total 471.39 last 205.60) - open/close 40000 (time 1746072882.06 total 476.11 last 123.68) - open/close 41774 (time 1746072892.06 total 486.11 last 177.34) - open/close 43375 (time 1746072902.06 total 496.12 last 160.02) - open/close 45183 (time 1746072912.07 total 506.12 last 180.74) - open/close 46742 (time 1746072922.08 total 516.13 last 155.77) - open/close 48155 (time 1746072932.09 total 526.14 last 141.13) - open/close 49603 (time 1746072942.09 total 536.14 last 144.79) total: 50000 open/close in 539.14 seconds: 92.74 ops/second - unlinked 0 (time 1746072946 ; total 0 ; last 0) - unlinked 10000 (time 1746072976 ; total 30 ; last 30) - unlinked 20000 (time 1746073013 ; total 67 ; last 37) - unlinked 30000 (time 1746073048 ; total 102 ; last 35) - unlinked 40000 (time 1746073084 ; total 138 ; last 36) total: 50000 unlinks in 178 seconds: 280.898865 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg234-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg234-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg234-server stop mds service on oleg234-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg234-server unloading modules via unload_modules_local on: 'oleg234-server' oleg234-server: oleg234-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg234-client: oleg234-client: ssh exited with exit code 2 pdsh@oleg234-client: oleg234-server: ssh exited with exit code 2 pdsh@oleg234-client: oleg234-client: ssh exited with exit code 2 pdsh@oleg234-client: oleg234-server: ssh exited with exit code 2