== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 18:09:57 (1746137397) start mds service on oleg235-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg235-server' oleg235-server: oleg235-server.virtnet: executing load_modules_local oleg235-server: Loading modules from /home/green/git/lustre-release/lustre oleg235-server: detected 4 online CPUs by sysfs oleg235-server: MODOPTS_LIBCFS= oleg235-server: Force libcfs to create 2 CPU partitions oleg235-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg235-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg235-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg235-server: oleg235-server.virtnet: executing set_default_debug -1 all pdsh@oleg235-client: oleg235-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg235-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg235-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg235-server: oleg235-server.virtnet: executing set_default_debug -1 all pdsh@oleg235-client: oleg235-server: ssh exited with exit code 1 Started lustre-OST0000 oleg235-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg235-client.virtnet: -o user_xattr,flock 192.168.202.135@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 253 (time 1746137440.23 total 10.67 last 23.71) - open/close 563 (time 1746137451.24 total 21.68 last 28.17) - open/close 922 (time 1746137461.24 total 31.68 last 35.88) - open/close 1221 (time 1746137472.08 total 42.52 last 27.58) - open/close 1390 (time 1746137483.43 total 53.87 last 14.90) - open/close 1611 (time 1746137494.29 total 64.73 last 20.35) - open/close 1954 (time 1746137505.26 total 75.70 last 31.24) - open/close 2264 (time 1746137515.84 total 86.28 last 29.32) - open/close 2476 (time 1746137526.75 total 97.19 last 19.42) - open/close 2674 (time 1746137537.50 total 107.94 last 18.42) - open/close 2929 (time 1746137548.21 total 118.65 last 23.81) - open/close 3170 (time 1746137558.98 total 129.42 last 22.37) - open/close 3426 (time 1746137569.30 total 139.74 last 24.81) - open/close 3748 (time 1746137579.30 total 149.74 last 32.20) - open/close 3994 (time 1746137590.24 total 160.68 last 22.49) - open/close 4183 (time 1746137600.24 total 170.68 last 18.89) - open/close 4404 (time 1746137610.24 total 180.68 last 22.10) - open/close 4671 (time 1746137621.08 total 191.52 last 24.65) - open/close 4900 (time 1746137631.08 total 201.52 last 22.88) - open/close 5148 (time 1746137641.69 total 212.13 last 23.38) - open/close 5330 (time 1746137652.15 total 222.59 last 17.40) - open/close 5623 (time 1746137662.87 total 233.31 last 27.34) - open/close 5832 (time 1746137673.30 total 243.74 last 20.04) - open/close 6019 (time 1746137683.37 total 253.81 last 18.56) - open/close 6203 (time 1746137693.37 total 263.81 last 18.40) - open/close 6319 (time 1746137704.19 total 274.63 last 10.72) - open/close 6480 (time 1746137714.54 total 284.98 last 15.56) - open/close 6734 (time 1746137724.95 total 295.39 last 24.41) - open/close 6991 (time 1746137735.14 total 305.58 last 25.23) - open/close 7255 (time 1746137745.67 total 316.11 last 25.06) - open/close 7432 (time 1746137755.69 total 326.13 last 17.67) - open/close 7682 (time 1746137766.43 total 336.87 last 23.27) - open/close 7865 (time 1746137776.44 total 346.88 last 18.28) - open/close 8107 (time 1746137787.07 total 357.51 last 22.76) - open/close 8372 (time 1746137797.23 total 367.67 last 26.10) - open/close 8588 (time 1746137807.27 total 377.71 last 21.50) - open/close 8795 (time 1746137817.39 total 387.83 last 20.46) - open/close 9085 (time 1746137827.40 total 397.84 last 28.98) - open/close 9638 (time 1746137837.40 total 407.84 last 55.29) - open/close 10000 (time 1746137839.67 total 410.11 last 159.68) - open/close 11432 (time 1746137849.67 total 420.11 last 143.18) - open/close 12851 (time 1746137859.67 total 430.11 last 141.88) - open/close 14239 (time 1746137869.68 total 440.12 last 138.63) - open/close 15766 (time 1746137879.68 total 450.12 last 152.68) - open/close 17271 (time 1746137889.68 total 460.12 last 150.47) - open/close 19100 (time 1746137899.69 total 470.13 last 182.87) - open/close 20000 (time 1746137904.29 total 474.73 last 195.54) - open/close 21783 (time 1746137914.29 total 484.73 last 178.25) - open/close 23469 (time 1746137924.29 total 494.73 last 168.54) - open/close 24939 (time 1746137934.30 total 504.74 last 146.92) - open/close 26429 (time 1746137944.31 total 514.75 last 148.85) - open/close 27714 (time 1746137954.31 total 524.75 last 128.46) - open/close 29213 (time 1746137964.32 total 534.76 last 149.86) - open/close 30000 (time 1746137969.59 total 540.03 last 149.20) - open/close 31788 (time 1746137979.59 total 550.03 last 178.80) - open/close 33561 (time 1746137989.59 total 560.03 last 177.22) - open/close 35654 (time 1746137999.60 total 570.04 last 209.19) - open/close 36887 (time 1746138009.60 total 580.04 last 123.27) - open/close 38481 (time 1746138019.61 total 590.05 last 159.35) - open/close 40000 (time 1746138029.00 total 599.44 last 161.62) - open/close 41438 (time 1746138039.01 total 609.45 last 143.75) - open/close 42754 (time 1746138049.01 total 619.45 last 131.60) - open/close 44484 (time 1746138059.01 total 629.45 last 172.96) - open/close 46300 (time 1746138069.02 total 639.46 last 181.52) - open/close 48316 (time 1746138079.02 total 649.46 last 201.59) total: 50000 open/close in 657.42 seconds: 76.06 ops/second - unlinked 0 (time 1746138088 ; total 0 ; last 0) - unlinked 10000 (time 1746138139 ; total 51 ; last 51) - unlinked 20000 (time 1746138183 ; total 95 ; last 44) - unlinked 30000 (time 1746138229 ; total 141 ; last 46) - unlinked 40000 (time 1746138279 ; total 191 ; last 50) total: 50000 unlinks in 235 seconds: 212.765961 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg235-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg235-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg235-server stop mds service on oleg235-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg235-server unloading modules via unload_modules_local on: 'oleg235-server' oleg235-server: oleg235-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg235-client: oleg235-client: ssh exited with exit code 2 pdsh@oleg235-client: oleg235-server: ssh exited with exit code 2 pdsh@oleg235-client: oleg235-client: ssh exited with exit code 2 pdsh@oleg235-client: oleg235-server: ssh exited with exit code 2