== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 23:56:59 (1733806619) start mds service on oleg301-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg301-server' oleg301-server: oleg301-server.virtnet: executing load_modules_local oleg301-server: Loading modules from /home/green/git/lustre-release/lustre oleg301-server: detected 4 online CPUs by sysfs oleg301-server: MODOPTS_LIBCFS= oleg301-server: Force libcfs to create 2 CPU partitions oleg301-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg301-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg301-server: oleg301-server.virtnet: executing set_default_debug -1 all pdsh@oleg301-client: oleg301-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg301-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg301-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg301-server: oleg301-server.virtnet: executing set_default_debug -1 all pdsh@oleg301-client: oleg301-server: ssh exited with exit code 1 Started lustre-OST0000 oleg301-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg301-client.virtnet: -o user_xattr,flock 192.168.203.101@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 89 (time 1733806661.01 total 12.90 last 6.90) - open/close 368 (time 1733806671.92 total 23.81 last 25.58) - open/close 650 (time 1733806682.37 total 34.26 last 26.99) - open/close 877 (time 1733806692.42 total 44.31 last 22.58) - open/close 1148 (time 1733806702.86 total 54.75 last 25.97) - open/close 1351 (time 1733806713.56 total 65.45 last 18.96) - open/close 1694 (time 1733806723.57 total 75.46 last 34.29) - open/close 1947 (time 1733806735.92 total 87.81 last 20.49) - open/close 2230 (time 1733806746.11 total 98.00 last 27.76) - open/close 2419 (time 1733806759.02 total 110.91 last 14.64) - open/close 2761 (time 1733806769.91 total 121.79 last 31.41) - open/close 3079 (time 1733806779.91 total 131.80 last 31.78) - open/close 3272 (time 1733806789.92 total 141.81 last 19.29) - open/close 3525 (time 1733806800.14 total 152.02 last 24.76) - open/close 3745 (time 1733806810.29 total 162.18 last 21.67) - open/close 4005 (time 1733806820.71 total 172.60 last 24.94) - open/close 4146 (time 1733806831.73 total 183.62 last 12.79) - open/close 4372 (time 1733806842.21 total 194.10 last 21.57) - open/close 4589 (time 1733806853.21 total 205.09 last 19.74) - open/close 4855 (time 1733806864.22 total 216.11 last 24.16) - open/close 5103 (time 1733806874.27 total 226.15 last 24.68) - open/close 5295 (time 1733806884.69 total 236.58 last 18.42) - open/close 5542 (time 1733806894.69 total 246.58 last 24.69) - open/close 5762 (time 1733806904.70 total 256.59 last 21.99) - open/close 6022 (time 1733806915.69 total 267.57 last 23.66) - open/close 6260 (time 1733806925.87 total 277.76 last 23.36) - open/close 6535 (time 1733806936.54 total 288.43 last 25.79) - open/close 6806 (time 1733806946.94 total 298.83 last 26.05) - open/close 7089 (time 1733806957.35 total 309.24 last 27.18) - open/close 7374 (time 1733806967.68 total 319.57 last 27.60) - open/close 7653 (time 1733806978.22 total 330.11 last 26.46) - open/close 7906 (time 1733806988.34 total 340.22 last 25.01) - open/close 8138 (time 1733806998.39 total 350.28 last 23.07) - open/close 8419 (time 1733807008.88 total 360.77 last 26.78) - open/close 8592 (time 1733807019.67 total 371.56 last 16.04) - open/close 8837 (time 1733807030.22 total 382.11 last 23.21) - open/close 9148 (time 1733807040.23 total 392.11 last 31.09) - open/close 9399 (time 1733807051.24 total 403.12 last 22.80) - open/close 9557 (time 1733807061.40 total 413.29 last 15.54) - open/close 9790 (time 1733807072.24 total 424.13 last 21.50) - open/close 10000 (time 1733807079.55 total 431.44 last 28.75) - open/close 12186 (time 1733807089.55 total 441.44 last 218.59) - open/close 14196 (time 1733807099.55 total 451.44 last 200.92) - open/close 16145 (time 1733807109.55 total 461.44 last 194.90) - open/close 17859 (time 1733807119.55 total 471.44 last 171.37) - open/close 19395 (time 1733807129.55 total 481.44 last 153.59) - open/close 20000 (time 1733807132.84 total 484.73 last 184.07) - open/close 21621 (time 1733807142.84 total 494.73 last 162.08) - open/close 23279 (time 1733807152.85 total 504.74 last 165.67) - open/close 24903 (time 1733807162.85 total 514.74 last 162.37) - open/close 26808 (time 1733807172.85 total 524.74 last 190.49) - open/close 28546 (time 1733807182.85 total 534.74 last 173.74) - open/close 30000 (time 1733807191.60 total 543.49 last 166.21) - open/close 31823 (time 1733807201.61 total 553.49 last 182.24) - open/close 33399 (time 1733807211.61 total 563.50 last 157.57) - open/close 35124 (time 1733807221.61 total 573.50 last 172.46) - open/close 36737 (time 1733807231.61 total 583.50 last 161.26) - open/close 38216 (time 1733807241.62 total 593.51 last 147.82) - open/close 39788 (time 1733807251.62 total 603.51 last 157.10) - open/close 41280 (time 1733807261.63 total 613.52 last 149.15) - open/close 42787 (time 1733807271.63 total 623.52 last 150.65) - open/close 44432 (time 1733807281.64 total 633.53 last 164.42) - open/close 45950 (time 1733807291.64 total 643.53 last 151.72) - open/close 47425 (time 1733807301.65 total 653.53 last 147.44) - open/close 48754 (time 1733807311.65 total 663.54 last 132.83) total: 50000 open/close in 672.60 seconds: 74.34 ops/second - unlinked 0 (time 1733807322 ; total 0 ; last 0) - unlinked 10000 (time 1733807366 ; total 44 ; last 44) - unlinked 20000 (time 1733807408 ; total 86 ; last 42) - unlinked 30000 (time 1733807445 ; total 123 ; last 37) - unlinked 40000 (time 1733807480 ; total 158 ; last 35) total: 50000 unlinks in 195 seconds: 256.410248 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg301-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg301-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg301-server stop mds service on oleg301-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg301-server unloading modules via unload_modules_local on: 'oleg301-server' oleg301-server: oleg301-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg301-client: oleg301-client: ssh exited with exit code 2 pdsh@oleg301-client: oleg301-server: ssh exited with exit code 2 pdsh@oleg301-client: oleg301-client: ssh exited with exit code 2 pdsh@oleg301-client: oleg301-server: ssh exited with exit code 2