== conf-sanity test 101a: Race MDT->OST reconnection with create ========================================================== 14:29:13 (1722796153) start mds service on oleg206-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg206-server' oleg206-server: oleg206-server.virtnet: executing load_modules_local oleg206-server: Loading modules from /home/green/git/lustre-release/lustre oleg206-server: detected 4 online CPUs by sysfs oleg206-server: Force libcfs to create 2 CPU partitions oleg206-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg206-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg206-server: oleg206-server.virtnet: executing set_default_debug -1 all pdsh@oleg206-client: oleg206-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg206-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg206-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg206-server: oleg206-server.virtnet: executing set_default_debug -1 all pdsh@oleg206-client: oleg206-server: ssh exited with exit code 1 Started lustre-OST0000 oleg206-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg206-client.virtnet: -o user_xattr,flock oleg206-server@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff - open/close 562 (time 1722796190.08 total 10.50 last 53.50) - open/close 1073 (time 1722796200.67 total 21.10 last 48.25) - open/close 1511 (time 1722796210.67 total 31.10 last 43.79) - open/close 1803 (time 1722796221.48 total 41.91 last 27.01) - open/close 2193 (time 1722796231.64 total 52.07 last 38.40) - open/close 2660 (time 1722796241.90 total 62.33 last 45.52) - open/close 2914 (time 1722796252.83 total 73.26 last 23.23) - open/close 3616 (time 1722796262.94 total 83.36 last 69.47) - open/close 4199 (time 1722796272.95 total 93.38 last 58.20) - open/close 4803 (time 1722796282.97 total 103.39 last 60.32) - open/close 5295 (time 1722796293.73 total 114.15 last 45.74) - open/close 5608 (time 1722796303.73 total 124.15 last 31.30) - open/close 5813 (time 1722796314.45 total 134.88 last 19.11) - open/close 6181 (time 1722796324.65 total 145.07 last 36.09) - open/close 6605 (time 1722796334.79 total 155.21 last 41.81) - open/close 7207 (time 1722796345.25 total 165.68 last 57.54) - open/close 7778 (time 1722796355.66 total 176.08 last 54.87) - open/close 8279 (time 1722796366.19 total 186.61 last 47.58) - open/close 8524 (time 1722796376.84 total 197.26 last 23.01) - open/close 8959 (time 1722796387.01 total 207.43 last 42.77) - open/close 9328 (time 1722796397.11 total 217.53 last 36.53) - open/close 9676 (time 1722796407.26 total 227.69 last 34.27) - open/close 10000 (time 1722796415.44 total 235.86 last 39.64) - open/close 10265 (time 1722796425.54 total 245.96 last 26.24) - open/close 10461 (time 1722796436.27 total 256.69 last 18.26) - open/close 10600 (time 1722796446.89 total 267.32 last 13.09) - open/close 10790 (time 1722796457.63 total 278.05 last 17.69) - open/close 11034 (time 1722796468.44 total 288.87 last 22.57) - open/close 11227 (time 1722796479.04 total 299.47 last 18.20) - open/close 13905 (time 1722796489.05 total 309.47 last 267.78) - open/close 17358 (time 1722796499.05 total 319.47 last 345.27) - open/close 20000 (time 1722796506.70 total 327.12 last 345.19) - open/close 22704 (time 1722796516.70 total 337.13 last 270.34) - open/close 25846 (time 1722796526.70 total 347.13 last 314.17) - open/close 29481 (time 1722796536.70 total 357.13 last 363.48) - open/close 33124 (time 1722796546.70 total 367.13 last 364.27) - open/close 36800 (time 1722796556.71 total 377.13 last 367.55) - open/close 40000 (time 1722796565.20 total 385.62 last 376.78) - open/close 43579 (time 1722796575.20 total 395.63 last 357.81) - open/close 47294 (time 1722796585.20 total 405.63 last 371.47) - open/close 49815 (time 1722796595.21 total 415.63 last 252.00) total: 50000 open/close in 416.38 seconds: 120.08 ops/second - unlinked 0 (time 1722796597 ; total 0 ; last 0) - unlinked 10000 (time 1722796612 ; total 15 ; last 15) - unlinked 20000 (time 1722796627 ; total 30 ; last 15) - unlinked 30000 (time 1722796643 ; total 46 ; last 16) - unlinked 40000 (time 1722796660 ; total 63 ; last 17) total: 50000 unlinks in 83 seconds: 602.409668 unlinks/second umount lustre on /mnt/lustre..... Stopping client oleg206-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg206-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg206-server stop mds service on oleg206-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg206-server unloading modules on: 'oleg206-server' oleg206-server: oleg206-server.virtnet: executing unload_modules_local modules unloaded. pdsh@oleg206-client: oleg206-server: ssh exited with exit code 2 pdsh@oleg206-client: oleg206-client: ssh exited with exit code 2 pdsh@oleg206-client: oleg206-client: ssh exited with exit code 2 pdsh@oleg206-client: oleg206-server: ssh exited with exit code 2