== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 19:24:12 (1746141852) start mds service on oleg207-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg207-server' oleg207-server: oleg207-server.virtnet: executing load_modules_local oleg207-server: Loading modules from /home/green/git/lustre-release/lustre oleg207-server: detected 4 online CPUs by sysfs oleg207-server: MODOPTS_LIBCFS= oleg207-server: Force libcfs to create 2 CPU partitions oleg207-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg207-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg207-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg207-server: oleg207-server.virtnet: executing set_default_debug -1 all pdsh@oleg207-client: oleg207-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg207-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg207-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg207-server: oleg207-server.virtnet: executing set_default_debug -1 all pdsh@oleg207-client: oleg207-server: ssh exited with exit code 1 Started lustre-OST0000 oleg207-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg207-client.virtnet: -o user_xattr,flock 192.168.202.107@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 240 (time 1746141894.58 total 10.22 last 23.49) - open/close 411 (time 1746141905.09 total 20.72 last 16.27) - open/close 690 (time 1746141915.16 total 30.80 last 27.70) - open/close 906 (time 1746141925.69 total 41.33 last 20.51) - open/close 1157 (time 1746141935.98 total 51.62 last 24.41) - open/close 1435 (time 1746141946.49 total 62.13 last 26.44) - open/close 1692 (time 1746141957.27 total 72.91 last 23.84) - open/close 1892 (time 1746141967.27 total 82.91 last 20.00) - open/close 2127 (time 1746141977.54 total 93.17 last 22.90) - open/close 2404 (time 1746141987.56 total 103.20 last 27.62) - open/close 2637 (time 1746141997.67 total 113.31 last 23.06) - open/close 2877 (time 1746142008.05 total 123.69 last 23.11) - open/close 3024 (time 1746142018.62 total 134.25 last 13.92) - open/close 3216 (time 1746142029.40 total 145.04 last 17.81) - open/close 3457 (time 1746142040.33 total 155.97 last 22.04) - open/close 3745 (time 1746142050.90 total 166.54 last 27.25) - open/close 3990 (time 1746142060.91 total 176.55 last 24.48) - open/close 4275 (time 1746142071.40 total 187.03 last 27.18) - open/close 4409 (time 1746142082.33 total 197.97 last 12.26) - open/close 4661 (time 1746142092.33 total 207.97 last 25.18) - open/close 5009 (time 1746142102.72 total 218.36 last 33.49) - open/close 5216 (time 1746142112.90 total 228.54 last 20.35) - open/close 5489 (time 1746142123.27 total 238.91 last 26.32) - open/close 5682 (time 1746142134.82 total 250.45 last 16.72) - open/close 5805 (time 1746142144.87 total 260.51 last 12.23) - open/close 6075 (time 1746142154.87 total 270.51 last 26.99) - open/close 6298 (time 1746142165.65 total 281.29 last 20.68) - open/close 6614 (time 1746142176.25 total 291.89 last 29.83) - open/close 6862 (time 1746142186.69 total 302.32 last 23.76) - open/close 7108 (time 1746142197.31 total 312.95 last 23.15) - open/close 7318 (time 1746142207.32 total 322.96 last 20.98) - open/close 7500 (time 1746142218.03 total 333.67 last 17.00) - open/close 7693 (time 1746142228.03 total 343.67 last 19.30) - open/close 7952 (time 1746142238.26 total 353.90 last 25.32) - open/close 8242 (time 1746142248.27 total 363.91 last 28.97) - open/close 8480 (time 1746142258.49 total 374.13 last 23.28) - open/close 8726 (time 1746142269.10 total 384.74 last 23.18) - open/close 8981 (time 1746142280.02 total 395.66 last 23.34) - open/close 9120 (time 1746142290.80 total 406.44 last 12.90) - open/close 9246 (time 1746142301.43 total 417.07 last 11.86) - open/close 9405 (time 1746142311.60 total 427.24 last 15.63) - open/close 9557 (time 1746142322.42 total 438.06 last 14.05) - open/close 10000 (time 1746142326.82 total 442.46 last 100.59) - open/close 11474 (time 1746142336.83 total 452.47 last 147.35) - open/close 12784 (time 1746142346.83 total 462.47 last 130.97) - open/close 14057 (time 1746142356.84 total 472.47 last 127.22) - open/close 15520 (time 1746142366.84 total 482.48 last 146.26) - open/close 16785 (time 1746142376.84 total 492.48 last 126.43) - open/close 18042 (time 1746142386.85 total 502.49 last 125.65) - open/close 19668 (time 1746142396.85 total 512.49 last 162.54) - open/close 20000 (time 1746142399.28 total 514.92 last 136.84) - open/close 21390 (time 1746142409.28 total 524.92 last 138.99) - open/close 22664 (time 1746142419.28 total 534.92 last 127.35) - open/close 23913 (time 1746142429.30 total 544.93 last 124.72) - open/close 25222 (time 1746142439.32 total 554.96 last 130.62) - open/close 26353 (time 1746142449.32 total 564.96 last 113.06) - open/close 27768 (time 1746142459.33 total 574.97 last 141.33) - open/close 29205 (time 1746142469.33 total 584.97 last 143.70) - open/close 30000 (time 1746142474.85 total 590.49 last 144.00) - open/close 31621 (time 1746142484.86 total 600.49 last 162.06) - open/close 33247 (time 1746142494.86 total 610.50 last 162.54) - open/close 34927 (time 1746142504.86 total 620.50 last 167.98) - open/close 36644 (time 1746142514.87 total 630.51 last 171.59) - open/close 38039 (time 1746142524.87 total 640.51 last 139.49) - open/close 39312 (time 1746142534.87 total 650.51 last 127.29) - open/close 40000 (time 1746142539.79 total 655.43 last 139.73) - open/close 41331 (time 1746142549.80 total 665.43 last 133.06) - open/close 42836 (time 1746142559.80 total 675.43 last 150.50) - open/close 44365 (time 1746142569.80 total 685.44 last 152.84) - open/close 45992 (time 1746142579.80 total 695.44 last 162.65) - open/close 47700 (time 1746142589.81 total 705.45 last 170.72) - open/close 49174 (time 1746142599.81 total 715.45 last 147.34) total: 50000 open/close in 721.53 seconds: 69.30 ops/second - unlinked 0 (time 1746142607 ; total 0 ; last 0) - unlinked 10000 (time 1746142665 ; total 58 ; last 58) - unlinked 20000 (time 1746142708 ; total 101 ; last 43) - unlinked 30000 (time 1746142751 ; total 144 ; last 43) - unlinked 40000 (time 1746142793 ; total 186 ; last 42) total: 50000 unlinks in 230 seconds: 217.391312 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg207-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg207-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg207-server stop mds service on oleg207-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg207-server unloading modules via unload_modules_local on: 'oleg207-server' oleg207-server: oleg207-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg207-client: oleg207-client: ssh exited with exit code 2 pdsh@oleg207-client: oleg207-server: ssh exited with exit code 2 pdsh@oleg207-client: oleg207-client: ssh exited with exit code 2 pdsh@oleg207-client: oleg207-server: ssh exited with exit code 2