== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 03:31:48 (1711092708) start mds service on oleg316-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg316-server' oleg316-server: oleg316-server.virtnet: executing load_modules_local oleg316-server: Loading modules from /home/green/git/lustre-release/lustre oleg316-server: detected 4 online CPUs by sysfs oleg316-server: Force libcfs to create 2 CPU partitions oleg316-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg316-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg316-server: oleg316-server.virtnet: executing set_default_debug -1 all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg316-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg316-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg316-server: oleg316-server.virtnet: executing set_default_debug -1 all pdsh@oleg316-client: oleg316-server: ssh exited with exit code 1 Started lustre-OST0000 oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg316-client.virtnet: -o user_xattr,flock oleg316-server@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 277 (time 1711092745.22 total 10.00 last 27.70) - open/close 712 (time 1711092755.32 total 20.10 last 43.07) - open/close 1241 (time 1711092765.77 total 30.55 last 50.61) - open/close 1707 (time 1711092776.77 total 41.55 last 42.38) - open/close 2222 (time 1711092786.89 total 51.67 last 50.86) - open/close 2586 (time 1711092796.90 total 61.68 last 36.38) - open/close 3001 (time 1711092806.94 total 71.73 last 41.31) - open/close 3506 (time 1711092817.41 total 82.19 last 48.25) - open/close 3935 (time 1711092828.03 total 92.81 last 40.41) - open/close 4369 (time 1711092838.35 total 103.13 last 42.05) - open/close 4790 (time 1711092848.88 total 113.66 last 39.98) - open/close 5194 (time 1711092859.62 total 124.40 last 37.60) - open/close 5379 (time 1711092870.42 total 135.20 last 17.14) - open/close 5778 (time 1711092881.04 total 145.83 last 37.55) - open/close 6111 (time 1711092891.60 total 156.38 last 31.55) - open/close 6408 (time 1711092901.91 total 166.69 last 28.81) - open/close 6690 (time 1711092912.06 total 176.84 last 27.78) - open/close 7104 (time 1711092922.67 total 187.45 last 39.01) - open/close 7443 (time 1711092933.45 total 198.23 last 31.45) - open/close 7723 (time 1711092946.03 total 210.81 last 22.27) - open/close 7995 (time 1711092956.24 total 221.02 last 26.64) - open/close 8275 (time 1711092966.67 total 231.45 last 26.83) - open/close 8598 (time 1711092977.26 total 242.04 last 30.52) - open/close 8949 (time 1711092987.61 total 252.39 last 33.91) - open/close 9211 (time 1711092997.73 total 262.51 last 25.89) - open/close 9584 (time 1711093008.35 total 273.13 last 35.11) - open/close 10000 (time 1711093016.33 total 281.11 last 52.12) - open/close 12404 (time 1711093026.33 total 291.12 last 240.38) - open/close 14474 (time 1711093036.34 total 301.12 last 206.91) - open/close 16092 (time 1711093046.34 total 311.13 last 161.70) - open/close 18174 (time 1711093056.35 total 321.13 last 208.11) - open/close 20000 (time 1711093065.11 total 329.90 last 208.28) - open/close 22330 (time 1711093075.12 total 339.90 last 232.94) - open/close 24304 (time 1711093085.12 total 349.91 last 197.27) - open/close 26014 (time 1711093095.12 total 359.91 last 170.98) - open/close 27972 (time 1711093105.13 total 369.91 last 195.74) - open/close 29606 (time 1711093115.13 total 379.91 last 163.33) - open/close 30000 (time 1711093118.36 total 383.14 last 122.20) - open/close 31048 (time 1711093128.36 total 393.14 last 104.76) - open/close 32448 (time 1711093138.36 total 403.15 last 139.95) - open/close 33768 (time 1711093148.37 total 413.15 last 131.95) - open/close 34990 (time 1711093158.38 total 423.16 last 122.11) - open/close 36082 (time 1711093168.38 total 433.16 last 109.14) - open/close 37223 (time 1711093178.39 total 443.17 last 114.04) - open/close 38283 (time 1711093188.40 total 453.18 last 105.89) - open/close 39215 (time 1711093198.40 total 463.18 last 93.16) - open/close 40000 (time 1711093206.27 total 471.05 last 99.75) - open/close 41115 (time 1711093216.28 total 481.06 last 111.44) - open/close 42062 (time 1711093226.28 total 491.06 last 94.69) - open/close 43104 (time 1711093236.28 total 501.06 last 104.17) - open/close 44264 (time 1711093246.28 total 511.06 last 116.00) - open/close 45401 (time 1711093256.29 total 521.07 last 113.63) - open/close 46295 (time 1711093266.29 total 531.07 last 89.37) - open/close 47157 (time 1711093276.30 total 541.08 last 86.09) - open/close 48156 (time 1711093286.31 total 551.09 last 99.80) - open/close 48975 (time 1711093296.32 total 561.10 last 81.82) - open/close 49840 (time 1711093306.33 total 571.11 last 86.42) total: 50000 open/close in 573.59 seconds: 87.17 ops/second - unlinked 0 (time 1711093310 ; total 0 ; last 0) - unlinked 10000 (time 1711093410 ; total 100 ; last 100) - unlinked 20000 (time 1711093505 ; total 195 ; last 95) - unlinked 30000 (time 1711093587 ; total 277 ; last 82) - unlinked 40000 (time 1711093738 ; total 428 ; last 151) total: 50000 unlinks in 477 seconds: 104.821800 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg316-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg316-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg316-server stop mds service on oleg316-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg316-server unloading modules on: 'oleg316-server' oleg316-server: oleg316-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg316-client: oleg316-client: ssh exited with exit code 2 pdsh@oleg316-client: oleg316-server: ssh exited with exit code 2 pdsh@oleg316-client: oleg316-client: ssh exited with exit code 2 pdsh@oleg316-client: oleg316-server: ssh exited with exit code 2