00010000:00010000:2.0F:1767316494.087807:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316495.115373:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0F:1767316496.136272:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0F:1767316496.530632:0:42482:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000020:00000040:0.0:1767316496.530656:0:42482:0:(genops.c:1229:class_import_get()) import ffff88c5c9ce5800 refcount=4 obd=MGC192.168.204.145@tcp 00000100:00000040:0.0:1767316496.530689:0:42482:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging cf740287-d7f4-4a51-9260-7c6190205bc2->MGS req@ffff88c5c6403480 x1853163991552768/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1767316496.530710:0:42482:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000e491838] to pc [ptlrpcd_00_01+1] req@ffff88c5c6403480 x1853163991552768/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.530745:0:42482:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000020:00000040:0.0:1767316496.530751:0:42482:0:(genops.c:1229:class_import_get()) import ffff88c5c6dc1000 refcount=4 obd=lustre-MDT0000-mdc-ffff88c5c68bd000 00000100:00000040:0.0:1767316496.530760:0:42482:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID req@ffff88c5c6400a80 x1853163991552896/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1767316496.530770:0:42482:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000621bc148] to pc [ptlrpcd_00_00+0] req@ffff88c5c6400a80 x1853163991552896/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.530778:0:42482:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000020:00000040:0.0:1767316496.530783:0:42482:0:(genops.c:1229:class_import_get()) import ffff88c5d1a0a800 refcount=3 obd=lustre-OST0000-osc-ffff88c5c68bd000 00000100:00000040:0.0:1767316496.530790:0:42482:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-OST0000_UUID req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1767316496.530799:0:42482:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000346c5a02] to pc [ptlrpcd_00_01+1] req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.530806:0:42482:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-OST0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000020:00000040:0.0:1767316496.530811:0:42482:0:(genops.c:1229:class_import_get()) import ffff88c5d1a0e800 refcount=3 obd=lustre-OST0001-osc-ffff88c5c68bd000 00000100:00000040:0.0:1767316496.530818:0:42482:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-OST0001_UUID req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000040:0.0:1767316496.530828:0:42482:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b49c3934] to pc [ptlrpcd_00_00+0] req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.530849:0:42482:0:(pinger.c:302:ptlrpc_pinger_main()) next wakeup in 5 (1700) 00000100:00000040:0.0:1767316496.533499:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff88c5c6403480 x1853163991552768/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316496.533522:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991552768:192.168.204.145@tcp:400:kworker.0 00000100:00000040:0.0:1767316496.533562:0:2487:0:(niobuf.c:1116:ptl_send_rpc()) @@@ send flags=200 req@ffff88c5c6403480 x1853163991552768/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316512 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.533570:0:2487:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.145@tcp 00000100:00000040:0.0:1767316496.533688:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316496.533698:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553024:192.168.204.145@tcp:400:kworker.0 00000100:00000040:0.0:1767316496.533712:0:2487:0:(niobuf.c:1116:ptl_send_rpc()) @@@ send flags=200 req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.533718:0:2487:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.145@tcp 00000100:00000040:0.0:1767316496.533731:0:2487:0:(niobuf.c:634:kick_cpu_latency()) lustre-OST0000-osc-ffff88c5c68bd000: using a timeout of 59731 usecs (60 jiffies) 00000100:00100000:0.0:1767316496.533747:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000040:0.0:1767316496.533754:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff88c5c6400a80 x1853163991552896/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316496.533762:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991552896:192.168.204.145@tcp:400:kworker.0 00000100:00000040:0.0:1767316496.533774:0:2487:0:(niobuf.c:1116:ptl_send_rpc()) @@@ send flags=200 req@ffff88c5c6400a80 x1853163991552896/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 1767316512 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.533781:0:2487:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.145@tcp 00000100:00000040:0.0:1767316496.533792:0:2487:0:(niobuf.c:634:kick_cpu_latency()) lustre-MDT0000-mdc-ffff88c5c68bd000: using a timeout of 154016 usecs (155 jiffies) 00000100:00000040:0.0:1767316496.533799:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316496.533819:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553152:192.168.204.145@tcp:400:kworker.0 00000100:00000040:0.0:1767316496.533833:0:2487:0:(niobuf.c:1116:ptl_send_rpc()) @@@ send flags=200 req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.533839:0:2487:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.204.145@tcp 00000100:00000040:0.0:1767316496.533852:0:2487:0:(niobuf.c:634:kick_cpu_latency()) lustre-OST0001-osc-ffff88c5c68bd000: using a timeout of 16745 usecs (17 jiffies) 00000100:00000040:3.0:1767316496.534091:0:2481:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff88c5c6403480 x1853163991552768/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316512 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1767316496.534144:0:2481:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1767316496.534160:0:2481:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff88c5c6400a80 x1853163991552896/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 1767316512 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1767316496.534208:0:2481:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1767316496.542967:0:2480:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff88c5c6400a80 x1853163991552896/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.543213:0:2487:0:(client.c:3077:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5c68bd000: skip recheck: last_committed 47244640404 00000100:00000040:0.0:1767316496.543233:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff88c5c6400a80 x1853163991552896/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.543245:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff88c5c6400a80 x1853163991552896/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316496.543253:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991552896:192.168.204.145@tcp:400:kworker.0 00000100:00000040:0.0:1767316496.543263:0:2487:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff88c5c6400a80 x1853163991552896/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1767316496.543276:0:2487:0:(genops.c:1242:class_import_put()) import ffff88c5c6dc1000 refcount=3 obd=lustre-MDT0000-mdc-ffff88c5c68bd000 00000100:00000040:3.0:1767316496.544269:0:2481:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1767316496.544307:0:2481:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1767316496.544330:0:2481:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff88c5c6403480 x1853163991552768/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00080000:0.0:1767316496.544585:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.544605:0:2487:0:(client.c:3077:ptlrpc_free_committed()) lustre-OST0001-osc-ffff88c5c68bd000: skip recheck: last_committed 4294968902 00000100:00000040:0.0:1767316496.544610:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.544618:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316496.544627:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553152:192.168.204.145@tcp:400:kworker.0 00000100:00000040:0.0:1767316496.544650:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff88c5c6403480 x1853163991552768/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.544658:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff88c5c6403480 x1853163991552768/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316496.544666:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991552768:192.168.204.145@tcp:400:kworker.0 00010000:00080000:0.0:1767316496.544677:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.544684:0:2487:0:(client.c:3077:ptlrpc_free_committed()) lustre-OST0000-osc-ffff88c5c68bd000: skip recheck: last_committed 8589934598 00000100:00000040:0.0:1767316496.544688:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1767316496.544695:0:2487:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316496.544701:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553024:192.168.204.145@tcp:400:kworker.0 00000100:00000040:0.0:1767316496.544707:0:2487:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff88c5c6400e00 x1853163991553152/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1767316496.544716:0:2487:0:(genops.c:1242:class_import_put()) import ffff88c5d1a0e800 refcount=2 obd=lustre-OST0001-osc-ffff88c5c68bd000 00000100:00000040:0.0:1767316496.544724:0:2487:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff88c5c6403480 x1853163991552768/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1767316496.544730:0:2487:0:(genops.c:1242:class_import_put()) import ffff88c5c9ce5800 refcount=3 obd=MGC192.168.204.145@tcp 00000100:00000040:0.0:1767316496.544735:0:2487:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff88c5c6402a00 x1853163991553024/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316512 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000020:00000040:0.0:1767316496.544742:0:2487:0:(genops.c:1242:class_import_put()) import ffff88c5d1a0a800 refcount=2 obd=lustre-OST0000-osc-ffff88c5c68bd000 00000100:00000040:3.0:1767316496.897555:0:8955:0:(connection.c:75:cpu_latency_work()) work item of ffff88c5c6760b30 (cpu 0) has reached its deadline 4296363376, at 4296363386 00000100:00000040:2.0:1767316496.897732:0:33179:0:(connection.c:75:cpu_latency_work()) work item of ffff88c5c6760a20 (cpu 0) has reached its deadline 4296363376, at 4296363386 00010000:00010000:1.0F:1767316497.161932:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316498.184264:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316499.209856:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316500.237377:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316501.256217:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316501.640164:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991553280:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316501.640316:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553536:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316501.640358:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1767316501.640365:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553408:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316501.640389:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553664:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316501.653255:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553408:192.168.204.145@tcp:400:kworker.0 00010000:00080000:1.0:1767316501.653283:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c65cf100 x1853163991553664/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316517 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316501.653296:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553664:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316501.653309:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991553280:192.168.204.145@tcp:400:kworker.0 00010000:00080000:1.0:1767316501.653321:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c65cf480 x1853163991553536/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316517 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316501.653328:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553536:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316502.279556:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316503.309714:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316504.328517:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316505.352579:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316506.376115:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1767316506.376356:0:42921:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 136: MDS to disconnect all OSPs first, then cleanup ldlm ========================================================== 20:15:04 (1767316504) 00000100:00100000:0.0:1767316506.762371:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991553792:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316506.762526:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991554048:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316506.762571:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1767316506.762578:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553920:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316506.762606:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991554176:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316506.772951:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991553920:192.168.204.145@tcp:400:kworker.0 00010000:00080000:0.0:1767316506.772979:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c6400a80 x1853163991554176/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316522 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316506.772992:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991554176:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316506.773006:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991553792:192.168.204.145@tcp:400:kworker.0 00010000:00080000:0.0:1767316506.773019:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c6400e00 x1853163991554048/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316522 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316506.773029:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991554048:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316507.411552:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1767316507.702025:0:43057:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: replay-single test_136 needs > 2 MDTs 00010000:00010000:2.0:1767316508.423552:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1767316509.173298:0:43209:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 137a: DNE: create under striped dir, fail MDT1 ========================================================== 20:15:07 (1767316507) 00010000:00010000:1.0:1767316509.451796:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1767316510.432787:0:43345:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: replay-single test_137a needs >= 2 MDTs 00010000:00010000:1.0:1767316510.473294:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316511.495332:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767316511.885687:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0a800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:02000000:3.0:1767316511.885696:0:8:0:(import.c:2080:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff88c5c68bd000: disconnect after 21s idle 00000100:00100000:2.0:1767316511.889675:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991554304:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316511.889891:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316511.889904:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991554432:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316511.897707:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991554560:192.168.204.145@tcp:9:kworker.0 00000100:00080000:3.0:1767316511.897921:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:02000000:3.0:1767316511.897932:0:8:0:(import.c:2080:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff88c5c68bd000: disconnect after 21s idle 00000100:00100000:3.0:1767316511.897960:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991554688:192.168.204.145@tcp:9:kworker.0 00000100:00100000:3.0:1767316511.900901:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991554304:192.168.204.145@tcp:400:kworker.0 00010000:00080000:1.0:1767316511.907236:0:2485:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c5729180 x1853163991554688/t0(0) o9->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316521 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316511.907258:0:2485:0:(import.c:1990:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff88c5c5729180 x1853163991554688/t0(0) o9->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316521 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316511.907273:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1767316511.907278:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991554688:192.168.204.145@tcp:9:kworker.0 00010000:00080000:1.0:1767316511.918383:0:2485:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c5729f80 x1853163991554560/t0(0) o9->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316521 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316511.918398:0:2485:0:(import.c:1990:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff88c5c5729f80 x1853163991554560/t0(0) o9->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316521 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316511.918404:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0a800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1767316511.918407:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991554560:192.168.204.145@tcp:9:kworker.0 00000100:00100000:3.0:1767316511.920557:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991554432:192.168.204.145@tcp:400:kworker.0 00000001:02000400:2.0:1767316512.228564:0:43498:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 137b: DNE: create under striped dir, fail MDT2 ========================================================== 20:15:10 (1767316510) 00010000:00010000:2.0:1767316512.519354:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316513.549703:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1767316513.655002:0:43634:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: replay-single test_137b needs >= 2 MDTs 00010000:00010000:1.0:1767316514.567928:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316515.595949:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1767316515.754590:0:43786:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 137c: DNE: create under striped dir, fail MDT1/MDT2 ========================================================== 20:15:14 (1767316514) 00010000:00010000:0.0:1767316516.615718:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767316516.999860:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991554816:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316517.014749:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1767316517.014783:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991554944:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316517.019030:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991554944:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316517.023427:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991554816:192.168.204.145@tcp:400:kworker.0 00000001:02000400:3.0:1767316517.578484:0:43923:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: replay-single test_137c needs >= 2 MDTs 00010000:00010000:3.0:1767316517.640682:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316518.663662:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1767316519.099421:0:44075:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 200: Dropping one OBD_PING should not cause disconnect ========================================================== 20:15:17 (1767316517) 00010000:00010000:3.0:1767316519.688434:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1767316520.419232:0:44211:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: replay-single test_200 Need remote client 00010000:00010000:0.0:1767316520.713958:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316521.743259:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1767316522.022451:0:44364:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 201: MDT umount cascading disconnects timeouts ========================================================== 20:15:20 (1767316520) 00000100:00100000:0.0:1767316522.122193:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991555072:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316522.122343:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767316522.122351:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991555200:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316522.127133:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991555200:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316522.127154:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991555072:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316522.759445:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1767316523.136777:0:44503:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: replay-single test_201 needs >= 2 MDTs 00010000:00010000:3.0:1767316523.789196:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316524.808364:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1767316524.879005:0:44656:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 202: pfl replay should recovery layout generation ========================================================== 20:15:23 (1767316523) 00000080:00200000:2.0:1767316525.773661:0:44682:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316525.774451:0:44682:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316525.774477:0:44682:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316525.774498:0:44682:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316525.774519:0:44682:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316525.774532:0:44682:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316525.774537:0:44682:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0881c 00000080:00010000:2.0:1767316525.774546:0:44682:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0881c from it ffffb10b01c7bb58 00010000:00010000:2.0:1767316525.774550:0:44682:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316525.774561:0:44682:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316525.774585:0:44682:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316525.774607:0:44682:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=13 00000080:00200000:2.0:1767316525.774611:0:44682:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:2.0:1767316525.774627:0:44682:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d202.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316525.774708:0:44682:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 44682 initiator: MDT0 00010000:00010000:2.0:1767316525.777067:0:44682:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44682 initiator: MDT0 00010000:00010000:2.0:1767316525.777113:0:44682:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44682 initiator: MDT0 00000100:00100000:2.0:1767316525.777157:0:44682:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44682:1853163991555328:192.168.204.145@tcp:101:lfs.0 00000100:00100000:2.0:1767316525.777315:0:44682:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca189300 going to sleep for 11 seconds 00000100:00100000:0.0:1767316525.780714:0:44682:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44682:1853163991555328:192.168.204.145@tcp:101:lfs.0 00010000:00010000:0.0:1767316525.780749:0:44682:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44682 initiator: MDT0 00010000:00010000:0.0:1767316525.780767:0:44682:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 44682 initiator: MDT0 00010000:00010000:0.0:1767316525.780775:0:44682:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 44682 initiator: MDT0 00010000:00010000:0.0:1767316525.780783:0:44682:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 44682 initiator: MDT0 00010000:00010000:0.0:1767316525.780793:0:44682:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 44682 initiator: MDT0 00010000:00010000:0.0:1767316525.780801:0:44682:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c606be00 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:0.0:1767316525.780807:0:44682:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 44682 initiator: MDT0 00010000:00010000:0.0:1767316525.780815:0:44682:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 44682 initiator: MDT0 00010000:00010000:0.0:1767316525.780824:0:44682:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 44682 initiator: MDT0 00010000:00010000:0.0:1767316525.780832:0:44682:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1767316525.780842:0:44682:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 44682 initiator: MDT0 00010000:00010000:0.0:1767316525.780849:0:44682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08957 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 44682 initiator: MDT0 00000002:00100000:0.0:1767316525.780890:0:44682:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@ffff88c5c57d6680 x1853163991555328/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316541 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1767316525.780907:0:44682:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1767316525.780945:0:44682:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.780962:0:44682:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00010000:0.0:1767316525.780975:0:44682:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0881c from it ffffb10b01c7b940 00010000:00010000:0.0:1767316525.780979:0:44682:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.780989:0:44682:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316525.781141:0:44682:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316525.781159:0:44682:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316525.781166:0:44682:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.781175:0:44682:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316525.781181:0:44682:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316525.781187:0:44682:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316525.781191:0:44682:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0881c 00000080:00010000:0.0:1767316525.781197:0:44682:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0881c from it ffffb10b01c7bbe8 00010000:00010000:0.0:1767316525.781208:0:44682:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.781216:0:44682:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316525.781237:0:44682:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316525.781248:0:44682:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d202.replay-single, flags=16392 00000080:00200008:0.0:1767316525.781286:0:44682:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1767316525.781293:0:44682:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316525.781300:0:44682:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316525.781307:0:44682:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.781316:0:44682:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316525.781322:0:44682:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316525.781327:0:44682:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316525.781330:0:44682:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0881c 00000080:00010000:0.0:1767316525.781335:0:44682:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0881c from it ffffb10b01c7bcb8 00010000:00010000:0.0:1767316525.781338:0:44682:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.781346:0:44682:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200008:0.0:1767316525.781974:0:44682:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1767316525.781980:0:44682:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316525.781995:0:44682:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316525.782003:0:44682:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.782012:0:44682:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316525.782018:0:44682:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316525.782023:0:44682:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316525.782026:0:44682:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0881c 00000080:00010000:0.0:1767316525.782030:0:44682:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0881c from it ffffb10b01c7bcb8 00010000:00010000:0.0:1767316525.782033:0:44682:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.782041:0:44682:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316525.782215:0:44682:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316525.782224:0:44682:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316525.782231:0:44682:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.782239:0:44682:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316525.782246:0:44682:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316525.782249:0:44682:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316525.782252:0:44682:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0881c 00000080:00010000:0.0:1767316525.782256:0:44682:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0881c from it ffffb10b01c7bbb0 00010000:00010000:0.0:1767316525.782265:0:44682:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.782274:0:44682:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316525.782282:0:44682:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 44 00000080:00200008:0.0:1767316525.782290:0:44682:0:(file.c:1001:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 300000 00000080:00200080:0.0:1767316525.782310:0:44682:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c03066f0 arg=7ffeeb524af0 00000080:00200000:0.0:1767316525.782323:0:44682:0:(dir.c:483:ll_dir_setdirstripe()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) name=d202.replay-single stripe_offset=0 stripe_count=1, hash_type=0 00010000:00010000:0.0:1767316525.782350:0:44682:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.782415:0:44682:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316525.782422:0:44682:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1767316525.782424:0:44682:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e5200/0x5295060e33d0881c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677892b expref: -99 pid: 40471 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316525.782450:0:44682:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316525.782462:0:44682:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6400e00 x1853163991555584/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 568/600 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316525.782488:0:44682:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44682:1853163991555584:192.168.204.145@tcp:36:lfs.0 00000100:00100000:0.0:1767316525.782605:0:44682:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426c00 going to sleep for 11 seconds 00000100:00100000:0.0:1767316525.796629:0:44682:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44682:1853163991555584:192.168.204.145@tcp:36:lfs.0 00000080:00200000:0.0:1767316525.796696:0:44682:0:(llite_lib.c:3217:ll_read_inode2()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00200000:0.0:1767316525.796704:0:44682:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 512 00000080:00200008:0.0:1767316525.796767:0:44682:0:(file.c:424:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 300000 00000080:00200000:2.0:1767316525.821345:0:44683:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316525.821384:0:44683:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316525.821735:0:44683:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 44683 initiator: MDT0 00010000:00010000:2.0:1767316525.821758:0:44683:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 44683 initiator: MDT0 00010000:00010000:2.0:1767316525.821770:0:44683:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 44683 initiator: MDT0 00000100:00100000:2.0:1767316525.821803:0:44683:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991555712:192.168.204.145@tcp:101:lfs.0 00000100:00100000:2.0:1767316525.822649:0:44683:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca189300 going to sleep for 11 seconds 00000100:00100000:1.0:1767316525.830431:0:44683:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991555712:192.168.204.145@tcp:101:lfs.0 00010000:00010000:1.0:1767316525.830468:0:44683:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.830482:0:44683:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.830520:0:44683:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.830531:0:44683:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.830545:0:44683:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.830553:0:44683:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.830560:0:44683:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316525.830572:0:44683:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d6d80 x1853163991555712/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316541 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316525.830587:0:44683:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.830601:0:44683:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1767316525.830622:0:44683:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316525.830656:0:44683:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:1.0:1767316525.830663:0:44683:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0895e 00000080:00010000:1.0:1767316525.830668:0:44683:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316525.830672:0:44683:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0895e from it ffffb10b0148bb58 00010000:00010000:1.0:1767316525.830675:0:44683:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.830689:0:44683:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316525.830721:0:44683:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316525.830746:0:44683:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:1.0:1767316525.830750:0:44683:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:1.0:1767316525.830779:0:44683:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d202.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316525.830835:0:44683:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.830843:0:44683:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.830854:0:44683:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44683 initiator: MDT0 00000100:00100000:1.0:1767316525.830879:0:44683:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991555840:192.168.204.145@tcp:101:lfs.0 00000100:00100000:1.0:1767316525.830995:0:44683:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00010000:00010000:1.0:1767316525.831655:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316525.841678:0:44683:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991555840:192.168.204.145@tcp:101:lfs.0 00010000:00010000:1.0:1767316525.841713:0:44683:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842051:0:44683:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842072:0:44683:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842171:0:44683:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842183:0:44683:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842198:0:44683:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842207:0:44683:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842215:0:44683:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316525.842230:0:44683:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cf480 x1853163991555840/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316541 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316525.842244:0:44683:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842256:0:44683:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1127 (0 0) 00000080:00200000:1.0:1767316525.842264:0:44683:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316525.842294:0:44683:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 512 00000080:00200000:1.0:1767316525.842298:0:44683:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x467:0x0](ffff88c5c997ba88): rc = 0 00000080:00010000:1.0:1767316525.842304:0:44683:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) for lock 0x5295060e33d08965 00000080:00010000:1.0:1767316525.842320:0:44683:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00010000:1.0:1767316525.842324:0:44683:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08965 from it ffffb10b0148bb20 00010000:00010000:1.0:1767316525.842334:0:44683:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842342:0:44683:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316525.842397:0:44683:0:(file.c:6252:ll_getattr_dentry()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316525.842406:0:44683:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88),name=d202.replay-single 00000002:00010000:1.0:1767316525.842435:0:44683:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316525.842445:0:44683:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842458:0:44683:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316525.842465:0:44683:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316525.842470:0:44683:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00010000:1.0:1767316525.842473:0:44683:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) for lock 0x5295060e33d08965 00000080:00010000:1.0:1767316525.842477:0:44683:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08965 from it ffffb10b0148bcb8 00010000:00010000:1.0:1767316525.842479:0:44683:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842486:0:44683:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316525.842560:0:44683:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316525.842568:0:44683:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316525.842575:0:44683:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842598:0:44683:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316525.842605:0:44683:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316525.842623:0:44683:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316525.842626:0:44683:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0895e 00000080:00010000:1.0:1767316525.842631:0:44683:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0895e from it ffffb10b0148bbf8 00010000:00010000:1.0:1767316525.842634:0:44683:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.842641:0:44683:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316525.842651:0:44683:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316525.842660:0:44683:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d202.replay-single, flags=267 00000080:00200000:1.0:1767316525.842666:0:44683:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 44 00000080:00200008:1.0:1767316525.842677:0:44683:0:(file.c:1001:ll_file_open()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 300000 00000002:00010000:1.0:1767316525.842694:0:44683:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0240300001 00010000:00010000:1.0:1767316525.842754:0:44683:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.842761:0:44683:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44683 initiator: MDT0 00000020:00100000:1.0:1767316525.842773:0:44683:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316525.842780:0:44683:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cf480 x1853163991556096/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 672/131880 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316525.842789:0:44683:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44683 initiator: MDT0 00000100:00100000:1.0:1767316525.842820:0:44683:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991556096:192.168.204.145@tcp:101:lfs.0 00000100:00100000:1.0:1767316525.843090:0:44683:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316525.852485:0:44683:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991556096:192.168.204.145@tcp:101:lfs.0 00010000:00010000:1.0:1767316525.852521:0:44683:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.852537:0:44683:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.852545:0:44683:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.852553:0:44683:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.852563:0:44683:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.852570:0:44683:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5d191bc00 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:1.0:1767316525.852576:0:44683:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.852584:0:44683:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.852591:0:44683:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.852600:0:44683:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1767316525.852608:0:44683:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 44683 initiator: MDT0 00010000:00010000:1.0:1767316525.853003:0:44683:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191bc00/0x5295060e33d0896c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 44683 initiator: MDT0 00000002:00100000:1.0:1767316525.853031:0:44683:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c65cf480 x1853163991556096/t47244640406(47244640406) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 672/608 e 0 to 0 dl 1767316541 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316525.853047:0:44683:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cf480 x1853163991556096/t47244640406(47244640406) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 672/608 e 0 to 0 dl 1767316541 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316525.853075:0:44683:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316525.853293:0:44683:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 512 00000002:00100000:1.0:1767316525.853317:0:44683:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cf480 x1853163991556096/t47244640406(47244640406) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 672/608 e 0 to 0 dl 1767316541 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316525.853677:0:44683:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 2 00000080:00200000:1.0:1767316525.853692:0:44683:0:(xattr.c:406:ll_xattr_set()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), xattr lov 00010000:00010000:1.0:1767316525.853722:0:44683:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.853790:0:44683:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316525.853797:0:44683:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316525.853800:0:44683:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d08965 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6778ffa expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000020:00100000:1.0:1767316525.853821:0:44683:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316525.853829:0:44683:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cc380 x1853163991556352/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 680/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316525.853854:0:44683:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991556352:192.168.204.145@tcp:36:lfs.0 00000100:00100000:1.0:1767316525.853970:0:44683:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184b180 going to sleep for 11 seconds 00000100:00100000:1.0:1767316525.859050:0:44683:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991556352:192.168.204.145@tcp:36:lfs.0 00000080:00200008:1.0:1767316525.859172:0:44683:0:(file.c:424:ll_file_release()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 300000 00000002:00100000:1.0:1767316525.859247:0:44683:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cf480 x1853163991556096/t47244640406(47244640406) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 672/608 e 0 to 0 dl 1767316541 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1767316525.859260:0:44683:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1767316525.859265:0:44683:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cf100 x1853163991556608/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/648 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1767316525.859292:0:44683:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991556608:192.168.204.145@tcp:35:lfs.0 00000100:00100000:1.0:1767316525.859451:0:44683:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316525.866955:0:44683:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44683:1853163991556608:192.168.204.145@tcp:35:lfs.0 00000002:00100000:1.0:1767316525.867000:0:44683:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cf480 x1853163991556096/t47244640406(47244640406) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 672/608 e 0 to 0 dl 1767316541 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200008:1.0:1767316526.754172:0:44692:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316526.754198:0:44692:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316526.754227:0:44692:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316526.754251:0:44692:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.754269:0:44692:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.754277:0:44692:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316526.754286:0:44692:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316526.754291:0:44692:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0895e 00000080:00010000:1.0:1767316526.754319:0:44692:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0895e from it ffffb10b019bfcb8 00010000:00010000:1.0:1767316526.754324:0:44692:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.754334:0:44692:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316526.757650:0:44692:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 0, flags 0, foreign 0 00000080:00200000:1.0:1767316526.757675:0:44692:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316526.757707:0:44692:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316526.757868:0:44692:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.757891:0:44692:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.757901:0:44692:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316526.757916:0:44692:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316526.757921:0:44692:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0895e 00000080:00010000:1.0:1767316526.757933:0:44692:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0895e from it ffffb10b019bfcb8 00010000:00010000:1.0:1767316526.757937:0:44692:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.757948:0:44692:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.758577:0:44692:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316526.758592:0:44692:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316526.758602:0:44692:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.758619:0:44692:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.758626:0:44692:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316526.758631:0:44692:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316526.758634:0:44692:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0895e 00000080:00010000:1.0:1767316526.758638:0:44692:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0895e from it ffffb10b019bfbb0 00010000:00010000:1.0:1767316526.758642:0:44692:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.758650:0:44692:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.758660:0:44692:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 44 00000080:00200008:1.0:1767316526.758682:0:44692:0:(file.c:1001:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 100000 00000080:00200080:1.0:1767316526.759187:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:1.0:1767316526.759213:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c65b9000 uarg=0000000000000000 00800000:00000004:1.0:1767316526.759221:0:44692:0:(obd_class.h:1083:obd_statfs()) lustre-MDT0000-mdc-ffff88c5c68bd000: age 1690, max_age 1724 00000100:00100000:1.0:1767316526.759286:0:44692:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44692:1853163991556736:192.168.204.145@tcp:41:lfs.0 00000100:00100000:1.0:1767316526.759448:0:44692:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316526.772087:0:44692:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44692:1853163991556736:192.168.204.145@tcp:41:lfs.0 00800000:00000004:1.0:1767316526.772151:0:44692:0:(obd_class.h:1109:obd_statfs()) lustre-MDT0000-mdc-ffff88c5c68bd000: update ffff88c5d0093520 cache blocks 17229/17271 objects 499431/499796 00000080:00200080:0.0:1767316526.772373:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772399:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772409:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772413:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772417:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772421:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772425:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772428:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772433:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772437:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772441:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772445:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772461:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772465:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772470:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772473:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772479:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772483:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772487:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772491:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772495:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772499:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772503:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772506:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772510:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772539:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772545:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772549:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772553:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772557:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772561:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772564:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772568:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772572:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772576:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772579:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772582:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772585:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772589:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772593:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772596:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772599:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772603:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772607:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772610:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772614:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772618:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772621:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772624:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772627:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772640:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772644:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772648:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772651:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772654:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772658:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772662:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772665:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772669:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772673:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772677:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772680:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772684:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772687:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772692:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772695:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772699:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772702:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772706:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772710:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772713:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772716:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772720:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772724:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772728:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772739:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772744:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772747:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772751:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772754:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772757:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772761:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772764:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772768:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772773:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772776:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772780:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772784:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772788:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772791:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772795:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772799:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772803:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772807:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772811:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772814:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772818:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772822:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772826:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772830:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772842:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772847:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772851:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772854:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772859:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772862:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772866:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772869:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772873:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772875:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772879:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772882:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772887:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772890:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772894:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772898:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772902:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772906:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772910:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772914:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772919:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772922:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772927:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772930:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772934:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772945:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772952:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00800000:00000080:0.0:1767316526.772956:0:44692:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.772960:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.772967:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00020000:00000004:0.0:1767316526.772978:0:44692:0:(obd_class.h:1083:obd_statfs()) lustre-OST0000-osc-ffff88c5c68bd000: age 1690, max_age 1724 00000100:00080000:0.0:1767316526.773000:0:44692:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1685 00000100:00080000:0.0:1767316526.773021:0:44692:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff88c5c68bd000: import ffff88c5d1a0a800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767316526.773038:0:44692:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767316526.773089:0:44692:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400a80 x1853163991556864/t0(0) o8->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767316526.773194:0:44692:0:(client.c:1829:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff88c5c6402a00 x1853163991556992/t0(0) o13->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316526.773206:0:44692:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426d80 going to sleep for 0 seconds 00000100:00100000:1.0:1767316526.773424:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991556864:192.168.204.145@tcp:8:lfs.0 00010000:00080000:1.0:1767316526.784685:0:2485:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c6400a80 x1853163991556864/t0(0) o8->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 520/416 e 0 to 0 dl 1767316536 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316526.784739:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-ffff88c5c68bd000: committing for last_committed 8589934598 gen 8 00000100:00080000:1.0:1767316526.784754:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff88c5c68bd000: connect to target with instance 4 00000100:00080000:1.0:1767316526.784766:0:2485:0:(import.c:951:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff88c5c68bd000: Resetting ns_connect_flags to server flags: 0xa0425af2e3440038 00000080:00000004:2.0:1767316526.784967:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:2.0:1767316526.784977:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:2.0:1767316526.784984:0:2485:0:(import.c:1204:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:2.0:1767316526.784990:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0a800 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:2.0:1767316526.784997:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:2.0:1767316526.785000:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:2.0:1767316526.785010:0:2485:0:(recover.c:189:ptlrpc_wake_delayed()) @@@ waking (set 00000000428b33b3): req@ffff88c5c6402a00 x1853163991556992/t0(0) o13->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316526.785051:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991556864:192.168.204.145@tcp:8:lfs.0 00000100:00100000:0.0:1767316526.785306:0:44692:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426d80 going to sleep for 11 seconds 00010000:00080000:0.0:1767316526.789391:0:44692:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c6402a00 x1853163991556992/t0(0) o13->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 1767316542 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316526.789423:0:44692:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44692:1853163991556992:192.168.204.145@tcp:13:lfs.0 00020000:00000004:0.0:1767316526.789648:0:44692:0:(obd_class.h:1109:obd_statfs()) lustre-OST0000-osc-ffff88c5c68bd000: update ffff88c5d0091c48 cache blocks 3678/3683 objects 117760/118186 00000080:00200080:0.0:1767316526.789748:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.789770:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00020000:00000004:0.0:1767316526.789779:0:44692:0:(obd_class.h:1083:obd_statfs()) lustre-OST0001-osc-ffff88c5c68bd000: age 1690, max_age 1724 00000100:00080000:0.0:1767316526.789795:0:44692:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1685 00000100:00080000:0.0:1767316526.789809:0:44692:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff88c5c68bd000: import ffff88c5d1a0e800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767316526.789841:0:44692:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767316526.789868:0:44692:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403480 x1853163991557120/t0(0) o8->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316526.789954:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991557120:192.168.204.145@tcp:8:lfs.0 00000100:00080000:0.0:1767316526.790994:0:44692:0:(client.c:1829:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff88c5c6402a00 x1853163991557248/t0(0) o13->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316526.791010:0:44692:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426d80 going to sleep for 0 seconds 00010000:00080000:3.0:1767316526.796298:0:2485:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c6403480 x1853163991557120/t0(0) o8->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 520/416 e 0 to 0 dl 1767316536 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767316526.796321:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-ffff88c5c68bd000: committing for last_committed 4294968902 gen 7 00000100:00080000:3.0:1767316526.796336:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff88c5c68bd000: connect to target with instance 2 00000100:00080000:3.0:1767316526.796350:0:2485:0:(import.c:951:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff88c5c68bd000: Resetting ns_connect_flags to server flags: 0xa0425af2e3440038 00000080:00000004:3.0:1767316526.796366:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:3.0:1767316526.796376:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:3.0:1767316526.796382:0:2485:0:(import.c:1204:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:3.0:1767316526.796387:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:3.0:1767316526.796414:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:3.0:1767316526.796417:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:3.0:1767316526.796427:0:2485:0:(recover.c:189:ptlrpc_wake_delayed()) @@@ waking (set 00000000428b33b3): req@ffff88c5c6402a00 x1853163991557248/t0(0) o13->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767316526.796448:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991557120:192.168.204.145@tcp:8:lfs.0 00000100:00100000:0.0:1767316526.796661:0:44692:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426d80 going to sleep for 11 seconds 00010000:00080000:0.0:1767316526.799560:0:44692:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c6402a00 x1853163991557248/t0(0) o13->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 1767316542 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316526.799586:0:44692:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44692:1853163991557248:192.168.204.145@tcp:13:lfs.0 00020000:00000004:0.0:1767316526.799618:0:44692:0:(obd_class.h:1109:obd_statfs()) lustre-OST0001-osc-ffff88c5c68bd000: update ffff88c5d0094df8 cache blocks 3674/3683 objects 117632/118571 00000080:00200080:0.0:1767316526.799774:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799795:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799803:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799807:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799811:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799815:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799819:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799823:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799827:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799831:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799835:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799839:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799843:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799868:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799874:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799878:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799881:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799884:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799888:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799891:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799894:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799897:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799901:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799904:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799907:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799911:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799914:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799918:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799922:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799926:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799929:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799933:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799937:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799941:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799945:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799949:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799952:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799955:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799967:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799971:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799976:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799979:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799983:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799987:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799992:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.799995:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.799999:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800002:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800007:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800011:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800015:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800018:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800022:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800024:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800028:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800031:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800035:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800038:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800053:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800057:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800062:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800065:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800069:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800080:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800085:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800089:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800133:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800140:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800146:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800150:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800153:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800157:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800160:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800164:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800168:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800171:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800175:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800178:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800181:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800196:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800202:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800206:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800210:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800214:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800217:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800221:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800225:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800229:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800242:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800248:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800252:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800256:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800260:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800264:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800268:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800271:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800276:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800280:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800284:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800287:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800291:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800295:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800299:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800302:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800306:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800310:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800315:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800318:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800323:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800326:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800329:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800332:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800336:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800347:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800352:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800366:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800372:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800375:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800379:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800383:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800386:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800390:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800394:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800397:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200080:0.0:1767316526.800401:0:44692:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd42a6f3f0 00020000:00000080:0.0:1767316526.800404:0:44692:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d3000 uarg=0000000000000000 00000080:00200008:0.0:1767316526.800423:0:44692:0:(file.c:424:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 100000 00010000:00010000:2.0:1767316526.857476:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1767316526.874228:0:44699:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316526.874273:0:44699:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316526.874297:0:44699:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316526.874323:0:44699:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316526.874333:0:44699:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316526.874345:0:44699:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316526.874349:0:44699:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0895e 00000080:00010000:0.0:1767316526.874454:0:44699:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0895e from it ffffb10b01c4bbd0 00010000:00010000:0.0:1767316526.874460:0:44699:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316526.874478:0:44699:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316526.874500:0:44699:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316526.874519:0:44699:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=fsa-oleg445-client.virtnet, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=1536 00000080:00200000:0.0:1767316526.874523:0:44699:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316526.874530:0:44699:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316526.874537:0:44699:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316526.874545:0:44699:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316526.874551:0:44699:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316526.874555:0:44699:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316526.874557:0:44699:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0895e 00000080:00010000:0.0:1767316526.874561:0:44699:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0895e from it ffffb10b01c4bcd0 00010000:00010000:0.0:1767316526.874564:0:44699:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316526.874571:0:44699:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316526.874579:0:44699:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1767316526.874585:0:44699:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316526.874592:0:44699:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316526.874597:0:44699:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316526.874613:0:44699:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316526.874619:0:44699:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316526.874623:0:44699:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316526.874626:0:44699:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d0895e 00000080:00010000:0.0:1767316526.874630:0:44699:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0895e from it ffffb10b01c4bdb0 00010000:00010000:0.0:1767316526.874633:0:44699:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316526.874640:0:44699:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316526.874647:0:44699:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1767316526.874653:0:44699:0:(namei.c:2065:ll_create_nd()) VFS Op:name=fsa-oleg445-client.virtnet, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=33188, excl=1 00000080:00200000:0.0:1767316526.874657:0:44699:0:(namei.c:2020:ll_mknod()) VFS Op:name=fsa-oleg445-client.virtnet, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) mode 100644 dev 0 00010000:00010000:0.0:1767316526.874690:0:44699:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316526.874756:0:44699:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316526.874763:0:44699:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1767316526.874767:0:44699:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ba00/0x5295060e33d0895e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6778fde expref: -99 pid: 44683 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316526.874798:0:44699:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316526.874815:0:44699:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6402a00 x1853163991557504/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 528/600 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316526.874853:0:44699:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job mcreate:5398f0ac-a698-45f7-862d-28236c50e3c9:44699:1853163991557504:192.168.204.145@tcp:36:mcreate.0 00000100:00100000:0.0:1767316526.874991:0:44699:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426d80 going to sleep for 11 seconds 00000100:00100000:0.0:1767316526.883264:0:44699:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job mcreate:5398f0ac-a698-45f7-862d-28236c50e3c9:44699:1853163991557504:192.168.204.145@tcp:36:mcreate.0 00000080:00200000:0.0:1767316526.883341:0:44699:0:(llite_lib.c:3217:ll_read_inode2()) VFS Op:inode=[0x200000bd1:0x468:0x0](ffff88c5c997b248) 00000080:00200000:0.0:1767316526.883409:0:44699:0:(namei.c:2073:ll_create_nd()) VFS Op:name=fsa-oleg445-client.virtnet, unhashed 1 00000080:00200000:0.0:1767316526.911005:0:44702:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316526.911058:0:44702:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316526.911878:0:44702:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 44702 initiator: MDT0 00010000:00010000:0.0:1767316526.911896:0:44702:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 44702 initiator: MDT0 00010000:00010000:0.0:1767316526.911910:0:44702:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 44702 initiator: MDT0 00000100:00100000:0.0:1767316526.911947:0:44702:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:44702:1853163991557632:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316526.912184:0:44702:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426d80 going to sleep for 11 seconds 00000100:00100000:1.0:1767316526.916524:0:44702:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:44702:1853163991557632:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316526.916560:0:44702:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.916575:0:44702:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.916590:0:44702:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.916602:0:44702:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.916615:0:44702:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.916624:0:44702:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.916632:0:44702:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316526.916644:0:44702:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6400700 x1853163991557632/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316542 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316526.916697:0:44702:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.916712:0:44702:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1767316526.916728:0:44702:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316526.916761:0:44702:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:1.0:1767316526.916770:0:44702:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08973 00000080:00010000:1.0:1767316526.916777:0:44702:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316526.916780:0:44702:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08973 from it ffffb10b01997b58 00010000:00010000:1.0:1767316526.916784:0:44702:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.916792:0:44702:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.916822:0:44702:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316526.916849:0:44702:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=fsa-oleg445-client.virtnet, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:1.0:1767316526.916854:0:44702:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=fsa-oleg445-client.virtnet, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:1.0:1767316526.916881:0:44702:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: fsa-oleg445-client.virtnet,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316526.916933:0:44702:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 44702 initiator: MDT0 00010000:00010000:1.0:1767316526.916940:0:44702:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44702 initiator: MDT0 00010000:00010000:1.0:1767316526.916950:0:44702:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44702 initiator: MDT0 00000100:00100000:1.0:1767316526.916974:0:44702:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:44702:1853163991557760:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316526.917128:0:44702:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316526.921025:0:44702:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:44702:1853163991557760:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316526.921069:0:44702:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921118:0:44702:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921127:0:44702:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921139:0:44702:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921152:0:44702:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921165:0:44702:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921174:0:44702:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921181:0:44702:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316526.921194:0:44702:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cf100 x1853163991557760/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 600/584 e 0 to 0 dl 1767316542 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316526.921208:0:44702:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921239:0:44702:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1128 (0 0) 00000080:00200000:1.0:1767316526.921247:0:44702:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316526.921274:0:44702:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x468:0x0], updating i_size 0 00000080:00200000:1.0:1767316526.921277:0:44702:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x468:0x0](ffff88c5c997b248): rc = 0 00000080:00200000:1.0:1767316526.921300:0:44702:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x468:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:1.0:1767316526.921307:0:44702:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x468:0x0](ffff88c5c997b248) for lock 0x5295060e33d0897a 00000080:00010000:1.0:1767316526.921321:0:44702:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x468:0x0](ffff88c5c997b248) 00000080:00010000:1.0:1767316526.921325:0:44702:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0897a from it ffffb10b01997b20 00010000:00010000:1.0:1767316526.921328:0:44702:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921337:0:44702:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316526.921388:0:44702:0:(file.c:6252:ll_getattr_dentry()) START file fsa-oleg445-client.virtnet:[0x200000bd1:0x468:0x0](ffff88c5c997b248), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316526.921396:0:44702:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x468:0x0](ffff88c5c997b248),name=fsa-oleg445-client.virtnet 00000002:00010000:1.0:1767316526.921422:0:44702:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x468:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316526.921431:0:44702:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921442:0:44702:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.921449:0:44702:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316526.921454:0:44702:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x468:0x0](ffff88c5c997b248) 00000080:00010000:1.0:1767316526.921458:0:44702:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x468:0x0](ffff88c5c997b248) for lock 0x5295060e33d0897a 00000080:00010000:1.0:1767316526.921462:0:44702:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0897a from it ffffb10b01997cb8 00010000:00010000:1.0:1767316526.921471:0:44702:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921478:0:44702:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.921562:0:44702:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316526.921571:0:44702:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316526.921578:0:44702:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921586:0:44702:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.921593:0:44702:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316526.921596:0:44702:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316526.921598:0:44702:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08973 00000080:00010000:1.0:1767316526.921602:0:44702:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08973 from it ffffb10b01997c10 00010000:00010000:1.0:1767316526.921605:0:44702:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.921611:0:44702:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.921619:0:44702:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316526.921631:0:44702:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=fsa-oleg445-client.virtnet, flags=0 00000080:00200000:1.0:1767316526.921636:0:44702:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316526.921642:0:44702:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316526.921647:0:44702:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.923441:0:44702:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.923556:0:44702:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316526.923573:0:44702:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316526.923579:0:44702:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08973 00000080:00010000:1.0:1767316526.923589:0:44702:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08973 from it ffffb10b01997d68 00010000:00010000:1.0:1767316526.923595:0:44702:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.923605:0:44702:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.923626:0:44702:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:1.0:1767316526.923641:0:44702:0:(namei.c:2354:ll_unlink()) VFS Op:name=fsa-oleg445-client.virtnet, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:1.0:1767316526.924015:0:44702:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.924066:0:44702:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316526.924079:0:44702:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x468:0x0]: losing layout lock 00010000:00010000:1.0:1767316526.924164:0:44702:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.924172:0:44702:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.924178:0:44702:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1767316526.924198:0:44702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b200/0x5295060e33d0897a lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x468:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779055 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316526.924218:0:44702:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341de00/0x5295060e33d08973 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779039 expref: -99 pid: 44702 timeout: 0 lvb_type: 0 00000020:00100000:1.0:1767316526.924246:0:44702:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316526.924263:0:44702:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cf100 x1853163991558016/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 512/648 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316526.924294:0:44702:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:44702:1853163991558016:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316526.927226:0:44702:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316526.938212:0:44702:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:44702:1853163991558016:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316526.938301:0:44702:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x468:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1767316526.938316:0:44702:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x468:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316526.938327:0:44702:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x468:0x0](ffff88c5c997b248) 00000100:00100000:1.0:1767316527.239813:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991558272:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316527.239984:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1767316527.239992:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991558144:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316527.240025:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991558400:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316527.249329:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991558144:192.168.204.145@tcp:400:kworker.0 00010000:00080000:1.0:1767316527.252306:0:2486:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c65cea00 x1853163991558400/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316543 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316527.252330:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991558400:192.168.204.145@tcp:400:kworker.0 00010000:00080000:1.0:1767316527.260389:0:2486:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c65cd880 x1853163991558272/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316543 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316527.260415:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991558272:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316527.879341:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316528.903302:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1767316529.434815:0:44732:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000 00000080:00200000:0.0:1767316529.473714:0:44733:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316529.473756:0:44733:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316529.473849:0:44733:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 44733 initiator: MDT0 00010000:00010000:0.0:1767316529.473862:0:44733:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 44733 initiator: MDT0 00010000:00010000:0.0:1767316529.473874:0:44733:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 44733 initiator: MDT0 00000100:00100000:0.0:1767316529.473920:0:44733:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991558528:192.168.204.145@tcp:101:touch.0 00000100:00100000:0.0:1767316529.474986:0:44733:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426b40 going to sleep for 11 seconds 00000100:00100000:0.0:1767316529.494049:0:44733:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991558528:192.168.204.145@tcp:101:touch.0 00010000:00010000:0.0:1767316529.494089:0:44733:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.494351:0:44733:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.494365:0:44733:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.494378:0:44733:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.494393:0:44733:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.494401:0:44733:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.494409:0:44733:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316529.494424:0:44733:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6402680 x1853163991558528/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316545 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316529.494442:0:44733:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.494736:0:44733:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1767316529.494746:0:44733:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316529.494787:0:44733:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:0.0:1767316529.494796:0:44733:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08981 00000080:00010000:0.0:1767316529.494805:0:44733:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316529.494808:0:44733:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08981 from it ffffb10b01c73bf8 00010000:00010000:0.0:1767316529.494812:0:44733:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.494821:0:44733:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316529.494860:0:44733:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316529.494888:0:44733:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=25 00000080:00200000:0.0:1767316529.494894:0:44733:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=0 00000002:00010000:0.0:1767316529.494920:0:44733:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d202.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:0.0:1767316529.495136:0:44733:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 44733 initiator: MDT0 00010000:00010000:0.0:1767316529.495146:0:44733:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44733 initiator: MDT0 00010000:00010000:0.0:1767316529.495157:0:44733:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44733 initiator: MDT0 00000100:00100000:0.0:1767316529.495186:0:44733:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991558656:192.168.204.145@tcp:101:touch.0 00000100:00100000:0.0:1767316529.495322:0:44733:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426b40 going to sleep for 11 seconds 00000100:00100000:0.0:1767316529.506252:0:44733:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991558656:192.168.204.145@tcp:101:touch.0 00010000:00010000:0.0:1767316529.506305:0:44733:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.507455:0:44733:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.507475:0:44733:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.507489:0:44733:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.507502:0:44733:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.507520:0:44733:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.507529:0:44733:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.507537:0:44733:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316529.507552:0:44733:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=10 disposition=b, status=0 req@ffff88c5c6402680 x1853163991558656/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316545 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316529.507565:0:44733:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.507591:0:44733:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1127 (0 0) 00000080:00200000:1.0:1767316529.507599:0:44733:0:(namei.c:955:ll_intent_lock()) intent lock 16 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316529.507628:0:44733:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 11776 00000080:00200000:1.0:1767316529.507632:0:44733:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x467:0x0](ffff88c5c997ba88): rc = 0 00000080:00010000:1.0:1767316529.507639:0:44733:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) for lock 0x5295060e33d08988 00000080:00010000:1.0:1767316529.507653:0:44733:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00010000:1.0:1767316529.507658:0:44733:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08988 from it ffffb10b01c73ac8 00010000:00010000:1.0:1767316529.507661:0:44733:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.507670:0:44733:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316529.507715:0:44733:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 1 00000080:00200000:1.0:1767316529.507736:0:44733:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 3 00000080:00200000:1.0:1767316529.507741:0:44733:0:(namei.c:1354:ll_atomic_open()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), file ffff88c5c4e56600, open_flags 8941, mode 81b6 opened 0 00000080:00200000:1.0:1767316529.507749:0:44733:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), intent=open|creat 00000002:00010000:1.0:1767316529.507777:0:44733:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f202.replay-single,[0x200000bd1:0x469:0x0]) in obj [0x200000bd1:0x467:0x0], intent: open|creat flags 0104502 00010000:00010000:1.0:1767316529.507830:0:44733:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 3/0,1 mode: --/CW res: [0x200000bd1:0x467:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 44733 initiator: MDT0 00010000:00010000:1.0:1767316529.507835:0:44733:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 3/0,1 mode: --/CW res: [0x200000bd1:0x467:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44733 initiator: MDT0 00000020:00100000:1.0:1767316529.507846:0:44733:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316529.507851:0:44733:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cd880 x1853163991558912/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 688/66344 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316529.507868:0:44733:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 3/0,1 mode: --/CW res: [0x200000bd1:0x467:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44733 initiator: MDT0 00000100:00100000:1.0:1767316529.507890:0:44733:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991558912:192.168.204.145@tcp:101:touch.0 00000100:00100000:1.0:1767316529.508188:0:44733:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316529.519293:0:44733:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991558912:192.168.204.145@tcp:101:touch.0 00010000:00010000:1.0:1767316529.519332:0:44733:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 4/0,1 mode: --/CW res: [0x200000bd1:0x467:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.519364:0:44733:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 4/0,1 mode: --/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.519372:0:44733:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 4/0,1 mode: --/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.519385:0:44733:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 4/0,1 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.519397:0:44733:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 4/0,1 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.519411:0:44733:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 5/1,1 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.519420:0:44733:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 5/1,1 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316529.519427:0:44733:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316529.519458:0:44733:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=3 disposition=37, status=0 req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 688/848 e 0 to 0 dl 1767316545 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316529.519471:0:44733:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 688/848 e 0 to 0 dl 1767316545 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316529.519489:0:44733:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 240 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316529.519502:0:44733:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316529.519515:0:44733:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1129 (0 0) 00000080:00200000:1.0:1767316529.519521:0:44733:0:(namei.c:955:ll_intent_lock()) intent lock 3 on i1 [0x200000bd1:0x467:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316529.519539:0:44733:0:(namei.c:1569:ll_create_it()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), intent=open|creat 00000080:00200000:1.0:1767316529.519575:0:44733:0:(llite_lib.c:3217:ll_read_inode2()) VFS Op:inode=[0x200000bd1:0x469:0x0](ffff88c5c85d21c8) 00000080:00200000:1.0:1767316529.519683:0:44733:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x469:0x0]: layout version change: 4294967294 -> 2 00000080:00010000:1.0:1767316529.519689:0:44733:0:(namei.c:1535:ll_create_node()) setting l_ast_data to inode [0x200000bd1:0x469:0x0](ffff88c5c85d21c8) 00000080:00010000:1.0:1767316529.519693:0:44733:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x469:0x0](ffff88c5c85d21c8) for lock 0x5295060e33d0898f 00000080:00010000:1.0:1767316529.519705:0:44733:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x469:0x0](ffff88c5c85d21c8) 00000080:00200008:1.0:1767316529.519725:0:44733:0:(file.c:1001:ll_file_open()) START file f202.replay-single:[0x200000bd1:0x469:0x0](ffff88c5c85d21c8), flags 104501 00000002:00100000:1.0:1767316529.519747:0:44733:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/848 e 0 to 0 dl 1767316545 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000080:00010000:1.0:1767316529.519790:0:44733:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d0898f from it ffff88c5c4d99c00 00010000:00010000:1.0:1767316529.519795:0:44733:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316529.519814:0:44733:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316529.519844:0:44733:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x469:0x0](ffff88c5c85d21c8), inode mode 81a4 mask 40 00000080:00200008:1.0:1767316529.519897:0:44733:0:(llite_lib.c:2314:ll_setattr_raw()) START file f202.replay-single:[0x200000bd1:0x469:0x0](ffff88c5c85d21c8) current size 0, valid attrs 20070, mode 88c5, uid 29834832, gid -20213, new size 18446744072418163223, atime 1767316529.000000000, mtime 1767316529.000000000, ctime 1767316529.000000000 00010000:00010000:1.0:1767316529.519922:0:44733:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316529.519939:0:44733:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x469:0x0]: losing layout lock 00010000:00010000:1.0:1767316529.520012:0:44733:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316529.520019:0:44733:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316529.520022:0:44733:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d0898f lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67790b7 expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316529.520051:0:44733:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316529.520057:0:44733:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cea00 x1853163991559168/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316529.520083:0:44733:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991559168:192.168.204.145@tcp:36:touch.0 00000100:00100000:1.0:1767316529.521368:0:44733:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316529.529006:0:44733:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991559168:192.168.204.145@tcp:36:touch.0 00000080:00200000:1.0:1767316529.529088:0:44733:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x469:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00010000:1.0:1767316529.529146:0:44733:0:(file.c:7031:ll_layout_intent()) ### lustre: requeue layout lock for file [0x200000bd1:0x469:0x0](ffff88c5c85d21c8) 00000002:00010000:1.0:1767316529.529160:0:44733:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x469:0x0]) in obj [0x200000bd1:0x469:0x0], intent: layout flags 0 00010000:00010000:1.0:1767316529.529257:0:44733:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 44733 initiator: MDT0 00010000:00010000:1.0:1767316529.529269:0:44733:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 44733 initiator: MDT0 00010000:00010000:1.0:1767316529.529280:0:44733:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 44733 initiator: MDT0 00000100:00100000:1.0:1767316529.529304:0:44733:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991559296:192.168.204.145@tcp:101:touch.0 00000100:00100000:1.0:1767316529.529421:0:44733:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316529.536395:0:44733:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991559296:192.168.204.145@tcp:101:touch.0 00010000:00010000:1.0:1767316529.536439:0:44733:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316529.536458:0:44733:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00000002:00100000:1.0:1767316529.536475:0:44733:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=400 disposition=1, status=0 req@ffff88c5c65cf480 x1853163991559296/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 376/584 e 0 to 0 dl 1767316545 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316529.536485:0:44733:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 240 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00000002:00010000:1.0:1767316529.536496:0:44733:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316529.536510:0:44733:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 16 res 8589937617/1129 (0 0) 00000080:00200000:1.0:1767316529.536519:0:44733:0:(namei.c:955:ll_intent_lock()) intent lock 1024 on i1 [0x200000bd1:0x469:0x0] suppgids 0 0: rc 0 00000080:00010000:1.0:1767316529.536547:0:44733:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x469:0x0](ffff88c5c85d21c8) for lock 0x5295060e33d08996 00000080:00010000:1.0:1767316529.536580:0:44733:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08996 from it ffffb10b01c73a78 00010000:00010000:1.0:1767316529.536584:0:44733:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316529.536593:0:44733:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316529.536615:0:44733:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 2/1,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316529.536625:0:44733:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 2/1,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00000080:00010000:1.0:1767316529.536631:0:44733:0:(file.c:6914:ll_layout_lock_set()) ### file [0x200000bd1:0x469:0x0](00000000b0f297d7) being reconfigured ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316529.536705:0:44733:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x469:0x0]: layout version change: 4294967294 -> 2 00010000:00010000:1.0:1767316529.536709:0:44733:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316529.536714:0:44733:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00020000:00200000:1.0:1767316529.536734:0:44733:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00020000:00200000:1.0:1767316529.536749:0:44733:0:(lov_io.c:860:lov_io_iter_init()) component[1] flags 0x0 00000100:00100000:0.0:1767316529.536849:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991559424:192.168.204.145@tcp:2:touch.0 00010000:00080000:0.0:1767316529.573533:0:2486:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c65cf480 x1853163991559424/t0(0) o2->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 440/432 e 0 to 0 dl 1767316545 ref 1 fl Rpc:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1767316529.573663:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991559424:192.168.204.145@tcp:2:touch.0 00000080:00200000:1.0:1767316529.573736:0:44733:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x469:0x0] ignore/verify layout 0/0, layout version 2 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:1.0:1767316529.573759:0:44733:0:(llite_lib.c:2564:ll_setattr_raw()) COMPLETED file f202.replay-single:[0x200000bd1:0x469:0x0](ffff88c5c85d21c8) current size 0, valid attrs 201f0, mode 88c5, uid 29834832, gid -20213, new size 18446744072418163223, atime 1767316529.000000000, mtime 1767316529.000000000, ctime 1767316529.000000000, rc 0 00000080:00200008:1.0:1767316529.573799:0:44733:0:(file.c:424:ll_file_release()) START file f202.replay-single:[0x200000bd1:0x469:0x0](ffff88c5c85d21c8), flags 104001 00000002:00100000:1.0:1767316529.573878:0:44733:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/848 e 0 to 0 dl 1767316545 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1767316529.573911:0:44733:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1767316529.573916:0:44733:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65ce300 x1853163991559680/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/736 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1767316529.573942:0:44733:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991559680:192.168.204.145@tcp:35:touch.0 00000100:00100000:0.0:1767316529.574571:0:44733:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1767316529.576806:0:44733:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job touch:5398f0ac-a698-45f7-862d-28236c50e3c9:44733:1853163991559680:192.168.204.145@tcp:35:touch.0 00000002:00100000:0.0:1767316529.576850:0:44733:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/848 e 0 to 0 dl 1767316545 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1767316529.603317:0:44735:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316529.603361:0:44735:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316529.603384:0:44735:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316529.603404:0:44735:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316529.603424:0:44735:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316529.603449:0:44735:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316529.603455:0:44735:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08981 00000080:00010000:2.0:1767316529.603463:0:44735:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08981 from it ffffb10b01997bf8 00010000:00010000:2.0:1767316529.603467:0:44735:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316529.603476:0:44735:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316529.603496:0:44735:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316529.603507:0:44735:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d202.replay-single, flags=27 00000080:00200000:2.0:1767316529.603513:0:44735:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 1 00000080:00200000:2.0:1767316529.603527:0:44735:0:(namei.c:1354:ll_atomic_open()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), file ffff88c5c9ff5680, open_flags 18800, mode 0 opened 0 00000080:00200000:2.0:1767316529.603535:0:44735:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), intent=open 00000002:00010000:2.0:1767316529.603553:0:44735:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f202.replay-single,[0x0:0x0:0x0]) in obj [0x200000bd1:0x467:0x0], intent: open flags 0304001 00010000:00010000:2.0:1767316529.603622:0:44735:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 44735 initiator: MDT0 00010000:00010000:2.0:1767316529.603630:0:44735:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44735 initiator: MDT0 00000020:00100000:2.0:1767316529.603645:0:44735:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1767316529.603662:0:44735:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d5180 x1853163991559936/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/131968 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1767316529.603672:0:44735:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44735 initiator: MDT0 00000100:00100000:2.0:1767316529.603714:0:44735:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44735:1853163991559936:192.168.204.145@tcp:101:lfs.0 00000100:00100000:2.0:1767316529.603984:0:44735:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca189300 going to sleep for 11 seconds 00000100:00100000:0.0:1767316529.611293:0:44735:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44735:1853163991559936:192.168.204.145@tcp:101:lfs.0 00010000:00010000:0.0:1767316529.611330:0:44735:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.611345:0:44735:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.611353:0:44735:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.611360:0:44735:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.611368:0:44735:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.611396:0:44735:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c606b400 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:0.0:1767316529.611401:0:44735:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.611408:0:44735:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.611414:0:44735:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 5/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.611421:0:44735:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1767316529.611428:0:44735:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.611433:0:44735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d0899d lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 44735 initiator: MDT0 00000002:00100000:0.0:1767316529.611455:0:44735:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=b, status=-20 req@ffff88c5c57d5180 x1853163991559936/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/608 e 0 to 0 dl 1767316545 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1767316529.612284:0:44735:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000bd1:0x467:0x0] suppgids 0 -1: rc -20 00000080:00200000:0.0:1767316529.612475:0:44735:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316529.612498:0:44735:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316529.612516:0:44735:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.612529:0:44735:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316529.612537:0:44735:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316529.612545:0:44735:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316529.612558:0:44735:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08981 00000080:00010000:0.0:1767316529.612565:0:44735:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08981 from it ffffb10b01997bf8 00010000:00010000:0.0:1767316529.612569:0:44735:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316529.612578:0:44735:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316529.612595:0:44735:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316529.612605:0:44735:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d202.replay-single, flags=267 00000080:00200000:0.0:1767316529.612614:0:44735:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 44 00000080:00200008:0.0:1767316529.612625:0:44735:0:(file.c:1001:ll_file_open()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 304000 00000002:00010000:0.0:1767316529.612642:0:44735:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100240304001 00010000:00010000:0.0:1767316529.612699:0:44735:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.612707:0:44735:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44735 initiator: MDT0 00000020:00100000:0.0:1767316529.612719:0:44735:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1767316529.612730:0:44735:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6403800 x1853163991560192/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/131968 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1767316529.612740:0:44735:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44735 initiator: MDT0 00000100:00100000:0.0:1767316529.612762:0:44735:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44735:1853163991560192:192.168.204.145@tcp:101:lfs.0 00000100:00100000:0.0:1767316529.613083:0:44735:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426b40 going to sleep for 11 seconds 00000100:00100000:0.0:1767316529.616290:0:44735:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44735:1853163991560192:192.168.204.145@tcp:101:lfs.0 00010000:00010000:0.0:1767316529.616322:0:44735:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.616358:0:44735:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.616366:0:44735:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.616374:0:44735:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.616384:0:44735:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.616391:0:44735:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c341da00 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:0.0:1767316529.616397:0:44735:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.616405:0:44735:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.616414:0:44735:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.616422:0:44735:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1767316529.616430:0:44735:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 44735 initiator: MDT0 00010000:00010000:0.0:1767316529.616437:0:44735:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341da00/0x5295060e33d089a4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 44735 initiator: MDT0 00000002:00100000:0.0:1767316529.616461:0:44735:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c6403800 x1853163991560192/t47244640414(47244640414) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316545 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1767316529.616477:0:44735:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6403800 x1853163991560192/t47244640414(47244640414) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316545 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1767316529.616500:0:44735:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316529.616527:0:44735:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 11776 00000002:00100000:0.0:1767316529.616547:0:44735:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6403800 x1853163991560192/t47244640414(47244640414) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316545 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200080:0.0:1767316529.616685:0:44735:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88) cmd=c0086915 arg=a3b3d0 00000100:00100000:0.0:1767316529.616766:0:44735:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44735:1853163991560320:192.168.204.145@tcp:34:lfs.0 00000100:00100000:0.0:1767316529.616941:0:44735:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426b40 going to sleep for 11 seconds 00000100:00100000:0.0:1767316529.630863:0:44735:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44735:1853163991560320:192.168.204.145@tcp:34:lfs.0 00000080:00200008:1.0:1767316529.632454:0:44735:0:(file.c:424:ll_file_release()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 304000 00000002:00100000:1.0:1767316529.632571:0:44735:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c6403800 x1853163991560192/t47244640414(47244640414) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316545 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1767316529.632585:0:44735:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1767316529.632591:0:44735:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cd500 x1853163991560576/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/736 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1767316529.632783:0:44735:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44735:1853163991560576:192.168.204.145@tcp:35:lfs.0 00000100:00100000:1.0:1767316529.632998:0:44735:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184b180 going to sleep for 11 seconds 00000100:00100000:1.0:1767316529.637532:0:44735:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:44735:1853163991560576:192.168.204.145@tcp:35:lfs.0 00000002:00100000:1.0:1767316529.637583:0:44735:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c6403800 x1853163991560192/t47244640414(47244640414) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316545 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00010000:00010000:1.0:1767316529.927706:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316530.951276:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316531.975289:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316532.360991:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991560704:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316532.361194:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991560960:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316532.361241:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1767316532.361248:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991560832:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316532.361276:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991561088:192.168.204.145@tcp:400:kworker.0 00010000:00080000:0.0:1767316532.372872:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c65cdc00 x1853163991561088/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316548 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316532.372899:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991561088:192.168.204.145@tcp:400:kworker.0 00010000:00080000:0.0:1767316532.372916:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c65cd180 x1853163991560960/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316548 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316532.372927:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991560960:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316533.000084:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316534.023731:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316535.047372:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316536.071697:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316537.095303:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316537.479432:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991561216:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316537.479439:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991561344:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316537.479560:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991561472:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316537.480621:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991561600:192.168.204.145@tcp:400:kworker.0 00010000:00080000:0.0:1767316537.488304:0:2486:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c65cc700 x1853163991561600/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316553 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316537.488325:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991561600:192.168.204.145@tcp:400:kworker.0 00010000:00080000:1.0:1767316537.490801:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c65ced80 x1853163991561472/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316553 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316537.490836:0:2487:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-ffff88c5c68bd000: committing for last_committed 12884901889 gen 8 00000100:00100000:1.0:1767316537.490866:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991561472:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316538.121199:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316539.144896:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316540.167628:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316541.191347:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316542.218221:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316542.600563:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991561728:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316542.600687:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991561984:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316542.600729:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1767316542.600737:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991561856:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316542.600776:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991562112:192.168.204.145@tcp:400:kworker.0 00010000:00080000:0.0:1767316542.607584:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c65cca80 x1853163991562112/t0(0) o400->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316558 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316542.607600:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991562112:192.168.204.145@tcp:400:kworker.0 00010000:00080000:0.0:1767316542.607608:0:2487:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c65cf800 x1853163991561984/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316558 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316542.607614:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991561984:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316543.240359:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316544.263560:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316545.290603:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316546.311261:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316547.336078:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316547.719889:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991562240:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316547.730379:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767316547.730415:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991562368:192.168.204.145@tcp:400:kworker.0 00000100:00080000:1.0:1767316547.731425:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:02000000:1.0:1767316547.731433:0:19117:0:(import.c:2080:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff88c5c68bd000: disconnect after 21s idle 00000100:00100000:0.0:1767316547.733959:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767316547.733999:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991562496:192.168.204.145@tcp:400:kworker.0 00010000:00080000:0.0:1767316547.746090:0:2486:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c65cdf80 x1853163991562496/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316563 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316547.746155:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991562496:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316547.759298:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991562624:192.168.204.145@tcp:9:kworker.0 00010000:00080000:0.0:1767316547.763889:0:2485:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c65ce680 x1853163991562624/t0(0) o9->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316557 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316547.763913:0:2485:0:(import.c:1990:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff88c5c65ce680 x1853163991562624/t0(0) o9->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316557 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316547.763923:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:0.0:1767316547.763928:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991562624:192.168.204.145@tcp:9:kworker.0 00010000:00010000:3.0:1767316548.359356:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:1.0:1767316548.743405:0:2487:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316532/real 1767316532] req@ffff88c5c65cce00 x1853163991560832/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316548 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316548.743441:0:2487:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c65cce00 x1853163991560832/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316548 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316548.743452:0:2487:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:1.0:1767316548.743456:0:2487:0:(import.c:181:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff88c5c68bd000: Connection to lustre-MDT0000 (at 192.168.204.145@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:1.0:1767316548.789627:0:2487:0:(import.c:416:ptlrpc_pinger_force()) lustre-MDT0000_UUID: waking up pinger s:DISCONN 00000100:00000400:1.0:1767316548.789715:0:2487:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316532/real 1767316532] req@ffff88c5c65cd500 x1853163991560704/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 1 dl 1767316548 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316548.789748:0:2487:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c65cd500 x1853163991560704/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 1 dl 1767316548 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316548.789758:0:2487:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from FULL to DISCONN 00000100:02020000:1.0:1767316548.789761:0:2487:0:(import.c:185:ptlrpc_set_import_discon()) MGC192.168.204.145@tcp: Connection to MGS (at 192.168.204.145@tcp) was lost; in progress operations using this service will fail 00000100:00080000:1.0:1767316548.810300:0:2487:0:(import.c:191:ptlrpc_set_import_discon()) import MGS@192.168.204.145@tcp for MGC192.168.204.145@tcp not replayable, auto-deactivating 00000100:00080000:1.0:1767316548.810306:0:2487:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:1.0:1767316548.810347:0:2487:0:(client.c:3559:ptlrpc_abort_inflight()) @@@ inflight req@ffff88c5c65cd500 x1853163991560704/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 1 dl 1767316548 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316548.810363:0:2487:0:(client.c:3559:ptlrpc_abort_inflight()) @@@ inflight req@ffff88c5c65cd180 x1853163991561216/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316553 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316548.810373:0:2487:0:(client.c:3559:ptlrpc_abort_inflight()) @@@ inflight req@ffff88c5c65ced80 x1853163991561728/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316558 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316548.810381:0:2487:0:(client.c:3559:ptlrpc_abort_inflight()) @@@ inflight req@ffff88c5c65cc000 x1853163991562240/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316563 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:1.0:1767316548.811451:0:2487:0:(mgc_request.c:1165:mgc_import_event()) import event 0x808001 00000100:00080000:1.0:1767316548.811457:0:2487:0:(pinger.c:192:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:1.0:1767316548.811459:0:2487:0:(mgc_request.c:1165:mgc_import_event()) import event 0x808002 00000100:00080000:1.0:1767316548.811461:0:2487:0:(import.c:416:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00080000:3.0:1767316548.811748:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316548.811754:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767316548.811758:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316548.811766:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1530 00000100:00080000:3.0:1767316548.811783:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767316548.811830:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729500 x1853163991562752/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767316548.811865:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316548.811869:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767316548.811871:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316548.811886:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1545 00000100:00080000:3.0:1767316548.811892:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767316548.811902:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767316548.811915:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572ad80 x1853163991562880/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316548.811947:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991562240:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316548.811958:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991561728:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316548.811968:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991561216:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316548.811985:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991563008:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316548.812049:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991560832:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316548.812057:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991560704:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316548.812161:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991562752:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767316548.812211:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991562880:192.168.204.145@tcp:38:kworker.0 00010000:00080000:3.0:1767316548.831137:0:2488:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c572aa00 x1853163991563008/t0(0) o400->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316564 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767316548.831160:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991563008:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316549.383328:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316550.408312:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767316550.859839:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316550.859865:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316550.859869:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316550.859873:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316550.859913:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0a800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:02000000:0.0:1767316550.859918:0:19117:0:(import.c:2080:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff88c5c68bd000: disconnect after 21s idle 00000100:00100000:1.0:1767316550.860023:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991563136:192.168.204.145@tcp:9:kworker.0 00010000:00080000:1.0:1767316550.880550:0:2485:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92325) req@ffff88c5c6401f80 x1853163991563136/t0(0) o9->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316560 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316550.880578:0:2485:0:(import.c:1990:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff88c5c6401f80 x1853163991563136/t0(0) o9->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316560 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316550.880588:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0a800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1767316550.880593:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991563136:192.168.204.145@tcp:9:kworker.0 00010000:00010000:3.0:1767316551.434262:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767316551.880556:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316551.880571:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316551.880596:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316551.880609:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767316552.455433:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316553.479497:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1767316553.863539:0:2486:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316537/real 1767316537] req@ffff88c5c65cdc00 x1853163991561344/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316553 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316553.863614:0:2486:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c65cdc00 x1853163991561344/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316553 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316553.863623:0:2486:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff88c5c6dc1000 already not connected (conn 36, was 35): CONNECTING 00000100:00100000:0.0:1767316553.863636:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991561344:192.168.204.145@tcp:400:kworker.0 00000100:00080000:1.0:1767316553.927921:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316553.927935:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316553.927939:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316553.927943:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767316554.504113:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316555.527465:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767316555.975522:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316555.975537:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316555.975541:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316555.975545:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767316556.551434:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316557.575704:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316558.600676:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1767316558.983601:0:2487:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316542/real 1767316542] req@ffff88c5c65cf480 x1853163991561856/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316558 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316558.983642:0:2487:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c65cf480 x1853163991561856/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316558 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316558.983653:0:2487:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff88c5c6dc1000 already not connected (conn 36, was 35): CONNECTING 00000100:00100000:0.0:1767316558.983675:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991561856:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316558.985336:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316548/real 1767316548] req@ffff88c5c5729500 x1853163991562752/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316558 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316558.985371:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729500 x1853163991562752/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316558 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316558.985384:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316548/real 1767316548] req@ffff88c5c572ad80 x1853163991562880/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316558 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316558.985397:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572ad80 x1853163991562880/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316558 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316558.985418:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316558.985445:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316558.985460:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991562752:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767316558.985467:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316558.985473:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316558.985476:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991562880:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767316558.986716:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316558.986728:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767316558.986733:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316558.986743:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1748 00000100:00080000:3.0:1767316558.986759:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1767316558.986780:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767316558.986832:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572aa00 x1853163991563264/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767316558.986885:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316558.986889:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767316558.986891:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316558.986896:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1748 00000100:00080000:3.0:1767316558.986902:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1767316558.986906:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767316558.986918:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767316558.986931:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729c00 x1853163991563392/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316558.988052:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991563264:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767316558.988249:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991563392:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767316558.995366:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) MGC192.168.204.145@tcp: connect to target with instance 0 10000000:01000000:1.0:1767316558.995383:0:2485:0:(mgc_request.c:1165:mgc_import_event()) import event 0x808005 00000100:02000400:1.0:1767316558.995392:0:2485:0:(import.c:1275:ptlrpc_connect_interpret()) Evicted from MGS (at 192.168.204.145@tcp) after server handle changed from 0x60377922a6777f45 to 0x60377922a6779127 00000100:00080000:1.0:1767316559.008509:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to EVICTED 00000100:00080000:1.0:1767316559.008526:0:2485:0:(import.c:1652:ptlrpc_import_recovery_state_machine()) evicted from MGS@192.168.204.145@tcp; invalidating 00000100:00100000:1.0:1767316559.010551:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991563264:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767316559.010634:0:45003:0:(import.c:1584:ptlrpc_invalidate_import_thread()) thread invalidate import MGC192.168.204.145@tcp to MGS@192.168.204.145@tcp 00000100:00100000:3.0:1767316559.010639:0:45003:0:(import.c:321:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 10000000:01000000:3.0:1767316559.010643:0:45003:0:(mgc_request.c:1165:mgc_import_event()) import event 0x808003 00010000:00010000:3.0:1767316559.010659:0:45003:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.145@tcp lock: ffff88c5c340ea00/0x5295060e33d086c5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110e400000000 nid: local remote: 0x60377922a677840e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:00010000:3.0:1767316559.010670:0:45003:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.145@tcp lock: ffff88c5c340ea00/0x5295060e33d086c5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x110f400000000 nid: local remote: 0x60377922a677840e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:01000000:3.0:1767316559.010676:0:45003:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1767316559.010694:0:45003:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1767316559.010703:0:45003:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.145@tcp lock: ffff88c5c340ea00/0x5295060e33d086c5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x590f400000000 nid: local remote: 0x60377922a677840e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316559.010716:0:45003:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.145@tcp lock: ffff88c5c340e800/0x5295060e33d086cc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x60377922a6778415 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:00010000:3.0:1767316559.010722:0:45003:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.145@tcp lock: ffff88c5c340e800/0x5295060e33d086cc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x60377922a6778415 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:01000000:3.0:1767316559.010741:0:45003:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1767316559.010745:0:45003:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1767316559.010748:0:45003:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.145@tcp lock: ffff88c5c340e800/0x5295060e33d086cc lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x60377922a6778415 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316559.010761:0:45003:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.145@tcp lock: ffff88c5c606b800/0x5295060e33d08950 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x60377922a6778f91 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:00010000:3.0:1767316559.010766:0:45003:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.145@tcp lock: ffff88c5c606b800/0x5295060e33d08950 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x60377922a6778f91 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:01000000:3.0:1767316559.010770:0:45003:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00010000:00010000:3.0:1767316559.010773:0:45003:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1767316559.010776:0:45003:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.145@tcp lock: ffff88c5c606b800/0x5295060e33d08950 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x60377922a6778f91 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316559.010789:0:45003:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.145@tcp lock: ffff88c5c3495600/0x5295060e33d0866a lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110e400000000 nid: local remote: 0x60377922a6777fb5 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:00010000:3.0:1767316559.010794:0:45003:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.145@tcp lock: ffff88c5c3495600/0x5295060e33d0866a lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x110f400000000 nid: local remote: 0x60377922a6777fb5 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:01000000:3.0:1767316559.010799:0:45003:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:3.0:1767316559.010802:0:45003:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1767316559.010805:0:45003:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.145@tcp lock: ffff88c5c3495600/0x5295060e33d0866a lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x590f400000000 nid: local remote: 0x60377922a6777fb5 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00080000:3.0:1767316559.010818:0:45003:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from EVICTED to RECOVER 00000100:00080000:3.0:1767316559.010823:0:45003:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from RECOVER to FULL 10000000:01000000:3.0:1767316559.010825:0:45003:0:(mgc_request.c:1165:mgc_import_event()) import event 0x808004 00000100:00080000:3.0:1767316559.010834:0:45003:0:(pinger.c:185:ptlrpc_pinger_ir_up()) IR up 00000100:02000000:3.0:1767316559.010838:0:45003:0:(import.c:1722:ptlrpc_import_recovery_state_machine()) MGC192.168.204.145@tcp: Connection restored to 192.168.204.145@tcp (at 192.168.204.145@tcp) 00010000:00010000:3.0:1767316559.623839:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:3.0:1767316560.519267:0:19117:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x469:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1767316560.519297:0:19117:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00020000:00200000:3.0:1767316560.519309:0:19117:0:(lov_io.c:860:lov_io_iter_init()) component[1] flags 0x0 00000080:00200000:3.0:1767316560.519330:0:19117:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x469:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00010000:00010000:3.0:1767316560.647404:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1767316560.647663:0:7230:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1767316560.647668:0:7230:0:(mgc_request.c:1812:mgc_process_log()) Process log lustre-cliir-ffff88c5c68bd000 from 20 10000000:01000000:1.0:1767316560.647674:0:7230:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1767316560.647732:0:7230:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.647744:0:7230:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.647756:0:7230:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316560.647786:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991563520:192.168.204.145@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1767316560.647893:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316560.674588:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991563520:192.168.204.145@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1767316560.674621:0:7230:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.674658:0:7230:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316560.678055:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991563648:192.168.204.145@tcp:256:ll_cfg_requeue.0 00000100:00100000:1.0:1767316560.678201:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316560.771667:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991563648:192.168.204.145@tcp:256:ll_cfg_requeue.0 10000000:01000000:1.0:1767316560.771724:0:7230:0:(mgc_request.c:1889:mgc_process_log()) MGC192.168.204.145@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1767316560.771733:0:7230:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.771745:0:7230:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:01000000:1.0:1767316560.771755:0:7230:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1767316560.771758:0:7230:0:(mgc_request.c:1812:mgc_process_log()) Process log lustre-client-ffff88c5c68bd000 from 55 10000000:01000000:1.0:1767316560.771761:0:7230:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1767316560.771795:0:7230:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5d191bc00/0x5295060e33d089b2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.771803:0:7230:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.145@tcp lock: ffff88c5d191bc00/0x5295060e33d089b2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.771812:0:7230:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.145@tcp lock: ffff88c5d191bc00/0x5295060e33d089b2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316560.771832:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991563776:192.168.204.145@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1767316560.771910:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316560.780486:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991563776:192.168.204.145@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1767316560.780527:0:7230:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.145@tcp lock: ffff88c5d191bc00/0x5295060e33d089b2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779135 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.780541:0:7230:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.145@tcp lock: ffff88c5d191bc00/0x5295060e33d089b2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a6779135 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316560.780620:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991563904:192.168.204.145@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1767316560.780697:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316560.783734:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991563904:192.168.204.145@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1767316560.783822:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564032:192.168.204.145@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1767316560.783965:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316560.786964:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564032:192.168.204.145@tcp:503:ll_cfg_requeue.0 00000040:00080000:1.0:1767316560.790751:0:45004:0:(llog.c:854:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 64768 count 54 00000020:01000000:0.0:1767316560.791752:0:7230:0:(obd_config.c:2043:class_config_parse_llog()) Processed log lustre-client gen 55-54 (rc=0) 10000000:01000000:0.0:1767316560.791782:0:7230:0:(mgc_request.c:1889:mgc_process_log()) MGC192.168.204.145@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1767316560.791796:0:7230:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5d191bc00/0x5295060e33d089b2 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779135 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316560.791831:0:7230:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.145@tcp lock: ffff88c5d191bc00/0x5295060e33d089b2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779135 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:01000000:0.0:1767316560.791842:0:7230:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:0.0:1767316560.791845:0:7230:0:(mgc_request.c:1812:mgc_process_log()) Process log params-ffff88c5c68bd000 from 7 10000000:01000000:0.0:1767316560.791848:0:7230:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1767316560.791984:0:7230:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341de00/0x5295060e33d089b9 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316560.791993:0:7230:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.145@tcp lock: ffff88c5c341de00/0x5295060e33d089b9 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316560.792597:0:7230:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.145@tcp lock: ffff88c5c341de00/0x5295060e33d089b9 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1767316560.792639:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564160:192.168.204.145@tcp:101:ll_cfg_requeue.0 00000100:00100000:0.0:1767316560.792809:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426300 going to sleep for 11 seconds 00000100:00100000:0.0:1767316560.795951:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564160:192.168.204.145@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1767316560.795991:0:7230:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.145@tcp lock: ffff88c5c341de00/0x5295060e33d089b9 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677913c expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316560.796344:0:7230:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.145@tcp lock: ffff88c5c341de00/0x5295060e33d089b9 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a677913c expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1767316560.796543:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564288:192.168.204.145@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1767316560.796701:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426300 going to sleep for 11 seconds 00000100:00100000:0.0:1767316560.806329:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564288:192.168.204.145@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1767316560.806448:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564416:192.168.204.145@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1767316560.806555:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426300 going to sleep for 11 seconds 00000100:00100000:0.0:1767316560.816940:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564416:192.168.204.145@tcp:503:ll_cfg_requeue.0 00000040:00080000:0.0:1767316560.818622:0:45005:0:(llog.c:854:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000020:01000000:1.0:1767316560.818868:0:7230:0:(obd_config.c:2043:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:1.0:1767316560.818903:0:7230:0:(mgc_request.c:1889:mgc_process_log()) MGC192.168.204.145@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:1.0:1767316560.818918:0:7230:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341de00/0x5295060e33d089b9 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677913c expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.818932:0:7230:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.145@tcp lock: ffff88c5c341de00/0x5295060e33d089b9 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677913c expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:01000000:1.0:1767316560.818943:0:7230:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:1.0:1767316560.818946:0:7230:0:(mgc_request.c:1812:mgc_process_log()) Process log lustre-sptlrpc-ffff88c5c500e360 from 1 10000000:01000000:1.0:1767316560.818949:0:7230:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:1.0:1767316560.819012:0:7230:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5d191b200/0x5295060e33d089c0 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.819050:0:7230:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.145@tcp lock: ffff88c5d191b200/0x5295060e33d089c0 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.819065:0:7230:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.145@tcp lock: ffff88c5d191b200/0x5295060e33d089c0 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316560.819137:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564544:192.168.204.145@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1767316560.819252:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316560.822606:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564544:192.168.204.145@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1767316560.822646:0:7230:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.145@tcp lock: ffff88c5d191b200/0x5295060e33d089c0 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779143 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.822661:0:7230:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.145@tcp lock: ffff88c5d191b200/0x5295060e33d089c0 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a6779143 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316560.822764:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564672:192.168.204.145@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1767316560.824157:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316560.835663:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991564672:192.168.204.145@tcp:501:ll_cfg_requeue.0 10000000:01000000:1.0:1767316560.835728:0:7230:0:(mgc_request.c:1889:mgc_process_log()) MGC192.168.204.145@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:1.0:1767316560.835741:0:7230:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5d191b200/0x5295060e33d089c0 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779143 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316560.835754:0:7230:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.145@tcp lock: ffff88c5d191b200/0x5295060e33d089c0 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779143 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316561.671366:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316562.695599:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1767316563.079517:0:2487:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316547/real 1767316547] req@ffff88c5c65cfb80 x1853163991562368/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316563 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316563.079559:0:2487:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c65cfb80 x1853163991562368/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316563 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316563.079569:0:2487:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff88c5c6dc1000 already not connected (conn 37, was 35): CONNECTING 00000100:00100000:0.0:1767316563.079591:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991562368:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316563.721681:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767316564.103367:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316564.103375:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316564.103395:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767316564.103419:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991564800:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316564.117405:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991564800:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316564.121354:0:2480:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.145@tcp (source: 12345-192.168.204.145@tcp) 00000100:00100000:3.0:1767316564.121423:0:7218:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1853164044867072 00000100:00100000:3.0:1767316564.121433:0:7218:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1853164044867072 00000100:00100000:3.0:1767316564.121490:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.145@tcp, seq: 1412 00000100:00100000:3.0:1767316564.121499:0:7218:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff88c5c57d7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:cf740287-d7f4-4a51-9260-7c6190205bc2+3:51426:x1853164044867072:12345-192.168.204.145@tcp:104: 00010000:00010000:3.0:1767316564.121515:0:7218:0:(ldlm_lockd.c:2505:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316564.121587:0:7218:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1767316564.121594:0:7218:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff88c5c57d7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:cf740287-d7f4-4a51-9260-7c6190205bc2+3:51426:x1853164044867072:12345-192.168.204.145@tcp:104: Request processed in 95us (247us total) trans 0 rc 0/0 00010000:00010000:0.0:1767316564.121630:0:15085:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1767316564.121639:0:15085:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316564.121739:0:15085:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5d191be00 already unused, calling callback (ffffffffc1436c20) 10000000:00010000:0.0:1767316564.121744:0:15085:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316564.121765:0:15085:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:00010000:0.0:1767316564.121771:0:15085:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 10000000:01000000:0.0:1767316564.121776:0:15085:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 00000100:00100000:3.0:1767316564.121803:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.145@tcp, seq: 1412 00010000:00010000:0.0:1767316564.121875:0:15085:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316564.121883:0:15085:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1767316564.121907:0:15085:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316564.121914:0:15085:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.145@tcp lock: ffff88c5d191be00/0x5295060e33d089ab lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x60377922a677912e expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316564.121932:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316564.121935:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316564.121937:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316564.121955:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991564928:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1767316564.124480:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991564928:192.168.204.145@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1767316564.743332:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316565.768882:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:1.0:1767316565.897619:0:7230:0:(mgc_request.c:611:do_requeue()) updating log lustre-cliir 10000000:01000000:1.0:1767316565.897630:0:7230:0:(mgc_request.c:1812:mgc_process_log()) Process log lustre-cliir-ffff88c5c68bd000 from 20 10000000:01000000:1.0:1767316565.897637:0:7230:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:1.0:1767316565.897709:0:7230:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5d191a600/0x5295060e33d089c7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316565.897723:0:7230:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.145@tcp lock: ffff88c5d191a600/0x5295060e33d089c7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316565.897736:0:7230:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.145@tcp lock: ffff88c5d191a600/0x5295060e33d089c7 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316565.897771:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991565056:192.168.204.145@tcp:101:ll_cfg_requeue.0 00000100:00100000:1.0:1767316565.897918:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316565.903744:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991565056:192.168.204.145@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1767316565.903787:0:7230:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.145@tcp lock: ffff88c5d191a600/0x5295060e33d089c7 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a67792b6 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316565.903804:0:7230:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.145@tcp lock: ffff88c5d191a600/0x5295060e33d089c7 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a67792b6 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316565.903916:0:7230:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991565184:192.168.204.145@tcp:256:ll_cfg_requeue.0 00000100:00100000:1.0:1767316565.904051:0:7230:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184bb40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316565.912847:0:7230:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:cf740287-d7f4-4a51-9260-7c6190205bc2:7230:1853163991565184:192.168.204.145@tcp:256:ll_cfg_requeue.0 00000020:00000080:1.0:1767316565.912914:0:7230:0:(obd_config.c:1389:class_process_config()) processing cmd: cf00f 10000000:01000000:1.0:1767316565.913140:0:7230:0:(mgc_request.c:1889:mgc_process_log()) MGC192.168.204.145@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:1.0:1767316565.913152:0:7230:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5d191a600/0x5295060e33d089c7 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a67792b6 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316565.913181:0:7230:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.145@tcp lock: ffff88c5d191a600/0x5295060e33d089c7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a67792b6 expref: -99 pid: 7230 timeout: 0 lvb_type: 0 00000100:00080000:0.0:1767316565.931696:0:45053:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff88c5c6dc1000 already not connected (conn 37, was 0): CONNECTING 00010000:00080000:0.0:1767316565.931741:0:45053:0:(ldlm_lib.c:85:import_set_conn()) imp ffff88c5c6dc1000@lustre-MDT0000-mdc-ffff88c5c68bd000: found existing conn 192.168.204.145@tcp, moved to head 00010000:00010000:3.0:1767316566.791589:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316567.815333:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316568.839240:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767316569.223510:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316569.223528:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316569.223558:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991565312:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316569.231969:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991565312:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316569.863393:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316570.887863:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316571.912297:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1767316572.092063:0:45106:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1767316572.092081:0:45106:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316572.092136:0:45106:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316572.092155:0:45106:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316572.092171:0:45106:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316572.092180:0:45106:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316572.092187:0:45106:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316572.092191:0:45106:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08981 00000080:00010000:2.0:1767316572.092198:0:45106:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08981 from it ffffb10b01fb3cb8 00010000:00010000:2.0:1767316572.092202:0:45106:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316572.092222:0:45106:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200008:2.0:1767316572.092752:0:45106:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 0, flags 0, foreign 0 00000080:00200000:2.0:1767316572.092757:0:45106:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316572.092766:0:45106:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316572.092778:0:45106:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316572.092788:0:45106:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316572.092795:0:45106:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316572.092800:0:45106:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316572.092803:0:45106:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08981 00000080:00010000:2.0:1767316572.092807:0:45106:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08981 from it ffffb10b01fb3cb8 00010000:00010000:2.0:1767316572.092811:0:45106:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316572.092819:0:45106:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316572.092903:0:45106:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316572.092911:0:45106:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316572.092918:0:45106:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316572.092927:0:45106:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316572.092934:0:45106:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316572.092938:0:45106:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316572.092941:0:45106:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08981 00000080:00010000:2.0:1767316572.092944:0:45106:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08981 from it ffffb10b01fb3bb0 00010000:00010000:2.0:1767316572.092947:0:45106:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316572.092960:0:45106:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316572.092969:0:45106:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 44 00000080:00200008:2.0:1767316572.092975:0:45106:0:(file.c:1001:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 100000 00000080:00200080:2.0:1767316572.093073:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316572.093088:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00800000:00000004:2.0:1767316572.093126:0:45106:0:(obd_class.h:1083:obd_statfs()) lustre-MDT0000-mdc-ffff88c5c68bd000: age 1725, max_age 1770 00000100:00080000:2.0:1767316572.093173:0:45106:0:(client.c:1829:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff88c5c57d4000 x1853163991565440/t0(0) o41->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316572.093185:0:45106:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c3474300 going to sleep for 0 seconds 00010000:00010000:0.0:1767316572.935298:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316573.127417:0:45106:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c3474300 going to sleep for 0 seconds 00000100:00100000:1.0:1767316573.319236:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316558/real 1767316558] req@ffff88c5c5729c00 x1853163991563392/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316573 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316573.319269:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729c00 x1853163991563392/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316573 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316573.319288:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316573.319315:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316573.319319:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991563392:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767316573.320441:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316573.320447:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767316573.320450:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316573.320457:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 0 00000100:00080000:2.0:1767316573.320470:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316573.320481:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316573.320510:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7b80 x1853163991565696/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316573.321417:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991565696:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767316573.321513:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2489:1853163991565568:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316573.327644:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2489:1853163991565568:192.168.204.145@tcp:400:kworker.0 00010000:00080000:1.0:1767316573.360068:0:2485:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c57d7b80 x1853163991565696/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/416 e 0 to 0 dl 1767316583 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316573.360085:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5c68bd000: committing for last_committed 47244640410 gen 2 00000100:00100000:1.0:1767316573.360141:0:2485:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/848 e 0 to 0 dl 1767316545 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1767316573.360157:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to target with instance 12 00000100:00080000:1.0:1767316573.360174:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to REPLAY 00000100:00080000:1.0:1767316573.360180:0:2485:0:(import.c:1679:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1767316573.360182:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5c68bd000: committing for last_committed 47244640410 gen 2 00000100:00100000:1.0:1767316573.360198:0:2485:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/848 e 0 to 0 dl 1767316545 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1767316573.360225:0:2485:0:(recover.c:53:ptlrpc_replay_next()) import ffff88c5c6dc1000 from lustre-MDT0000_UUID committed 47244640410 last 0 00000100:00080000:1.0:1767316573.360233:0:2485:0:(client.c:3523:ptlrpc_replay_req()) @@@ REPLAY req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/848 e 0 to 0 dl 1767316545 ref 1 fl New:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1767316573.360245:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991565696:192.168.204.145@tcp:38:kworker.0 00000100:00100000:1.0:1767316573.360268:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991558912:192.168.204.145@tcp:101:touch.0 00000100:00080000:1.0:1767316573.419528:0:2485:0:(client.c:3408:ptlrpc_replay_interpret()) @@@ got reply req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/608 e 0 to 0 dl 1767316589 ref 2 fl Interpret:RQU/604/0 rc 301/301 job:'touch.0' uid:0 gid:0 projid:0 00000002:00080000:1.0:1767316573.419553:0:2485:0:(mdc_request.c:714:mdc_replay_open()) @@@ updating close body with new fh req@ffff88c5c65ce300 x1853163991559680/t47244640413(47244640413) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/456 e 0 to 0 dl 1767316545 ref 2 fl Complete:RQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000100:00020000:1.0:1767316573.419563:0:2485:0:(client.c:3418:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/608 e 0 to 0 dl 1767316589 ref 2 fl Interpret:RQU/604/0 rc 301/301 job:'touch.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1767316573.456634:0:2485:0:(import.c:1679:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:1.0:1767316573.456641:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5c68bd000: committing for last_committed 47244640410 gen 2 00000100:00100000:1.0:1767316573.456654:0:2485:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/608 e 0 to 0 dl 1767316589 ref 2 fl Interpret:RQU/604/0 rc 301/301 job:'touch.0' uid:0 gid:0 projid:0 00000100:00080000:1.0:1767316573.456665:0:2485:0:(recover.c:53:ptlrpc_replay_next()) import ffff88c5c6dc1000 from lustre-MDT0000_UUID committed 47244640410 last 47244640411 00000100:00080000:1.0:1767316573.456674:0:2485:0:(client.c:3523:ptlrpc_replay_req()) @@@ REPLAY req@ffff88c5c65cea00 x1853163991559168/t47244640412(47244640412) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/456 e 0 to 0 dl 1767316545 ref 1 fl New:RQU/204/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316573.456687:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991558912:192.168.204.145@tcp:101:touch.0 00000100:00100000:2.0:1767316573.456759:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991559168:192.168.204.145@tcp:36:touch.0 00000100:00080000:2.0:1767316573.461873:0:2485:0:(client.c:3408:ptlrpc_replay_interpret()) @@@ got reply req@ffff88c5c65cea00 x1853163991559168/t47244640412(47244640412) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/456 e 0 to 0 dl 1767316589 ref 2 fl Interpret:RQU/204/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767316573.461888:0:2485:0:(import.c:1679:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1767316573.461904:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5c68bd000: committing for last_committed 47244640410 gen 2 00000100:00100000:2.0:1767316573.461910:0:2485:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/608 e 0 to 0 dl 1767316589 ref 1 fl Complete:RQU/604/0 rc 0/301 job:'touch.0' uid:0 gid:0 projid:0 00000100:00080000:2.0:1767316573.461918:0:2485:0:(recover.c:53:ptlrpc_replay_next()) import ffff88c5c6dc1000 from lustre-MDT0000_UUID committed 47244640410 last 47244640412 00000100:00080000:2.0:1767316573.461925:0:2485:0:(client.c:3523:ptlrpc_replay_req()) @@@ REPLAY req@ffff88c5c65ce300 x1853163991559680/t47244640413(47244640413) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/456 e 0 to 0 dl 1767316545 ref 2 fl New:RQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000100:00100000:2.0:1767316573.461938:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991559168:192.168.204.145@tcp:36:touch.0 00000100:00100000:2.0:1767316573.461958:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991559680:192.168.204.145@tcp:35:touch.0 00000100:00080000:2.0:1767316573.465821:0:2485:0:(client.c:3408:ptlrpc_replay_interpret()) @@@ got reply req@ffff88c5c65ce300 x1853163991559680/t47244640413(47244640413) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/456 e 0 to 0 dl 1767316589 ref 3 fl Interpret:RQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000100:00080000:2.0:1767316573.465835:0:2485:0:(import.c:1679:ptlrpc_import_recovery_state_machine()) replay requested by lustre-MDT0000_UUID 00000100:00100000:2.0:1767316573.465837:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5c68bd000: committing for last_committed 47244640410 gen 2 00000100:00100000:2.0:1767316573.465843:0:2485:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff88c5c65cd880 x1853163991558912/t47244640411(47244640411) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/608 e 0 to 0 dl 1767316589 ref 1 fl Complete:RQU/604/0 rc 0/301 job:'touch.0' uid:0 gid:0 projid:0 00000100:00080000:2.0:1767316573.465850:0:2485:0:(recover.c:53:ptlrpc_replay_next()) import ffff88c5c6dc1000 from lustre-MDT0000_UUID committed 47244640410 last 47244640413 00000100:00080000:2.0:1767316573.465854:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from REPLAY to REPLAY_LOCKS 00000100:00100000:0.0:1767316573.466333:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991559680:192.168.204.145@tcp:35:touch.0 00010000:00080000:2.0:1767316573.466391:0:45121:0:(ldlm_request.c:2714:ldlm_lock_replay_thread()) lock replay thread lustre-MDT0000-mdc-ffff88c5c68bd000 to lustre-MDT0000_UUID@192.168.204.145@tcp 00010000:00010000:2.0:1767316573.466395:0:45121:0:(ldlm_request.c:2634:ldlm_cancel_unused_locks_for_replay()) Dropping as many unused locks as possible before replay for namespace lustre-MDT0000-mdc-ffff88c5c68bd000 (29) 00010000:00010000:2.0:1767316573.466469:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08981 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0x60377922a677907f expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316573.466511:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x469:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466525:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08996 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x469:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a67790cc expref: -99 pid: 44733 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316573.466564:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08988 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0x60377922a677909b expref: -99 pid: 44733 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316573.466576:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x451:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466582:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08807 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x451:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a67788c9 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466775:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x450:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466786:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c6021e4000/0x5295060e33d087f9 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x450:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778898 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466802:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44f:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466807:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4aa00/0x5295060e33d087eb lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778867 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466831:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44d:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466837:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4b800/0x5295060e33d087cf lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778805 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466849:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44c:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466855:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d087c1 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a67787d4 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466869:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44a:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466874:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d087a5 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778772 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466886:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x449:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466901:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4b600/0x5295060e33d08797 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x449:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778741 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466914:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x448:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466920:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c340f200/0x5295060e33d08789 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x448:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778710 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466933:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x447:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466938:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4b000/0x5295060e33d0877b lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x447:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a67786df expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466950:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x446:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466955:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341dc00/0x5295060e33d0876d lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x446:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a67786ae expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466966:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x442:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466971:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c340ec00/0x5295060e33d08735 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x442:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a67785ea expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.466983:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x444:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.466988:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c400/0x5295060e33d08751 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x444:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a677864c expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.467000:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x445:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467005:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4a400/0x5295060e33d0875f lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x445:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a677867d expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.467017:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44e:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467022:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4ac00/0x5295060e33d087dd lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778836 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.467047:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x443:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467054:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c000/0x5295060e33d08743 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x443:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a677861b expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.467065:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44b:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467070:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c600/0x5295060e33d087b3 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a67787a3 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.467082:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x441:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467087:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4a000/0x5295060e33d08727 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x441:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a67785b9 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.467195:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x440:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467205:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c340ee00/0x5295060e33d08719 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x440:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778588 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.467217:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x43f:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467223:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d800/0x5295060e33d0870b lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778557 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316573.467236:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x43e:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467241:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d600/0x5295060e33d086fd lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778526 expref: -99 pid: 40352 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316573.467261:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c3494a00/0x5295060e33d086ef lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0x60377922a67784f5 expref: -99 pid: 40352 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316573.467272:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x43b:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467293:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4b200/0x5295060e33d086a9 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778382 expref: -99 pid: 38624 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316573.467317:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c3494000/0x5295060e33d086a2 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0x60377922a6778366 expref: -99 pid: 38624 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316573.467330:0:45121:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x43a:0x0]: losing layout lock 00010000:00010000:2.0:1767316573.467341:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4a800/0x5295060e33d08694 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000000 nid: local remote: 0x60377922a6778343 expref: -99 pid: 38624 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316573.467359:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c7d4bc00/0x5295060e33d0868d lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0x60377922a6778327 expref: -99 pid: 38624 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316573.467377:0:45121:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c340f800/0x5295060e33d08686 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000000 nid: local remote: 0x60377922a677830b expref: -99 pid: 38624 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316573.467387:0:45121:0:(ldlm_request.c:2647:ldlm_cancel_unused_locks_for_replay()) Canceled 0 unused locks from namespace lustre-MDT0000-mdc-ffff88c5c68bd000 00000100:00080000:2.0:1767316573.468243:0:45121:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from REPLAY_LOCKS to REPLAY_WAIT 00000100:00100000:1.0:1767316573.469060:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991565824:192.168.204.145@tcp:400:ldlm_lock_repla.0 00000100:00080000:1.0:1767316573.565496:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from REPLAY_WAIT to RECOVER 00000100:00080000:1.0:1767316573.565540:0:2485:0:(client.c:3203:ptlrpc_resend_req()) @@@ going to resend req@ffff88c5c57d4700 x1853163991565824/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 1767316589 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316573.565552:0:2485:0:(client.c:3212:ptlrpc_resend_req()) @@@ it has reply, so skip it req@ffff88c5c57d4700 x1853163991565824/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 1767316589 ref 1 fl Interpret:RQU/2c0/0 rc 0/0 job:'ldlm_lock_repla.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316573.565560:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:0.0:1767316573.565902:0:2485:0:(import.c:1722:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff88c5c68bd000: Connection restored to 192.168.204.145@tcp (at 192.168.204.145@tcp) 00000100:00080000:0.0:1767316573.565911:0:2485:0:(recover.c:189:ptlrpc_wake_delayed()) @@@ waking (set 00000000066db64c): req@ffff88c5c57d4000 x1853163991565440/t0(0) o41->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316573.565934:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991565824:192.168.204.145@tcp:400:ldlm_lock_repla.0 00000100:00100000:2.0:1767316573.566090:0:45106:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c3474300 going to sleep for 11 seconds 00000100:00100000:2.0:1767316573.571143:0:45106:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5c68bd000: committing for last_committed 47244640413 gen 2 00000100:00100000:2.0:1767316573.571190:0:45106:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45106:1853163991565440:192.168.204.145@tcp:41:lfs.0 00800000:00000004:2.0:1767316573.571211:0:45106:0:(obd_class.h:1109:obd_statfs()) lustre-MDT0000-mdc-ffff88c5c68bd000: update ffff88c5d0093520 cache blocks 17229/17271 objects 499700/500067 00000080:00200080:2.0:1767316573.571289:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571307:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571315:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571320:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571324:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571342:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571347:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571351:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571355:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571359:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571363:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571366:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571370:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571373:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571377:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571381:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571385:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571388:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571392:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571396:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571399:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571402:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571405:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571408:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571412:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571415:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571419:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571422:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571426:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571429:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571441:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571446:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571450:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571453:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571456:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571460:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571464:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571468:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571472:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571476:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571479:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571483:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571487:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571490:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571494:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571497:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571501:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571504:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571508:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571511:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571515:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571519:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571536:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571541:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571545:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571556:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571561:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571565:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571569:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571573:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571577:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571581:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571585:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571589:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571605:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571610:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571614:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571617:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571621:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571624:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571628:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571632:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571636:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571639:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571643:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571647:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571651:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571654:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571658:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571662:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571673:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571678:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571683:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571687:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571691:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571694:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571699:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571702:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571705:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571709:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571713:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571716:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571720:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571723:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571727:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571730:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571735:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571739:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571743:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571747:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571750:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571754:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571758:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571762:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571765:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571789:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571795:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571799:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571803:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571807:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571811:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571814:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571818:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571821:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571824:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571827:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571831:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571834:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571838:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571841:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571845:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571848:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571852:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571856:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571859:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571862:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571865:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00800000:00000080:2.0:1767316573.571869:0:45106:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.571873:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.571878:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00020000:00000004:2.0:1767316573.571891:0:45106:0:(obd_class.h:1083:obd_statfs()) lustre-OST0000-osc-ffff88c5c68bd000: age 1725, max_age 1771 00000100:00080000:2.0:1767316573.571905:0:45106:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1725 00000100:00080000:2.0:1767316573.571915:0:45106:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff88c5c68bd000: import ffff88c5d1a0a800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316573.571925:0:45106:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316573.571949:0:45106:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4700 x1853163991565952/t0(0) o8->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767316573.571988:0:45106:0:(client.c:1829:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff88c5c57d4000 x1853163991566080/t0(0) o13->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316573.571998:0:45106:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c351a600 going to sleep for 0 seconds 00000100:00100000:0.0:1767316573.572286:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991565952:192.168.204.145@tcp:8:lfs.0 00010000:00080000:0.0:1767316573.580451:0:2485:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c57d4700 x1853163991565952/t0(0) o8->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 520/416 e 0 to 0 dl 1767316583 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316573.580469:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-ffff88c5c68bd000: committing for last_committed 12884901889 gen 9 00000100:00080000:0.0:1767316573.580479:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff88c5c68bd000: connect to target with instance 4 00000100:00080000:0.0:1767316573.581076:0:2485:0:(import.c:951:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff88c5c68bd000: Resetting ns_connect_flags to server flags: 0xa0425af2e3440038 00000080:00000004:0.0:1767316573.581091:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:0.0:1767316573.581118:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1767316573.581124:0:2485:0:(import.c:1204:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:0.0:1767316573.581128:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0a800 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1767316573.581134:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:0.0:1767316573.581153:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1767316573.581258:0:2485:0:(recover.c:189:ptlrpc_wake_delayed()) @@@ waking (set 00000000e6e844f5): req@ffff88c5c57d4000 x1853163991566080/t0(0) o13->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316573.581277:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991565952:192.168.204.145@tcp:8:lfs.0 00000100:00100000:2.0:1767316573.586386:0:45106:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c351a600 going to sleep for 11 seconds 00010000:00080000:2.0:1767316573.592712:0:45106:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c57d4000 x1853163991566080/t0(0) o13->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 1767316589 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316573.592734:0:45106:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45106:1853163991566080:192.168.204.145@tcp:13:lfs.0 00020000:00000004:2.0:1767316573.592761:0:45106:0:(obd_class.h:1109:obd_statfs()) lustre-OST0000-osc-ffff88c5c68bd000: update ffff88c5d0091c48 cache blocks 3678/3683 objects 117760/118186 00000080:00200080:2.0:1767316573.592943:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.592959:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00020000:00000004:2.0:1767316573.592966:0:45106:0:(obd_class.h:1083:obd_statfs()) lustre-OST0001-osc-ffff88c5c68bd000: age 1725, max_age 1771 00000100:00080000:2.0:1767316573.592979:0:45106:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1725 00000100:00080000:2.0:1767316573.592989:0:45106:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff88c5c68bd000: import ffff88c5d1a0e800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316573.592998:0:45106:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316573.593019:0:45106:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7b80 x1853163991566208/t0(0) o8->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767316573.593062:0:45106:0:(client.c:1829:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff88c5c57d4000 x1853163991566336/t0(0) o13->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316573.593089:0:45106:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c351a600 going to sleep for 0 seconds 00000100:00100000:0.0:1767316573.593210:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991566208:192.168.204.145@tcp:8:lfs.0 00010000:00080000:0.0:1767316573.596983:0:2485:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c57d7b80 x1853163991566208/t0(0) o8->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 520/416 e 0 to 0 dl 1767316583 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316573.596998:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-ffff88c5c68bd000: committing for last_committed 4294968902 gen 8 00000100:00080000:0.0:1767316573.597008:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff88c5c68bd000: connect to target with instance 2 00000100:00080000:0.0:1767316573.597018:0:2485:0:(import.c:951:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff88c5c68bd000: Resetting ns_connect_flags to server flags: 0xa0425af2e3440038 00000080:00000004:0.0:1767316573.597028:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:0.0:1767316573.597034:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1767316573.597038:0:2485:0:(import.c:1204:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:0.0:1767316573.597042:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:0.0:1767316573.597047:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:0.0:1767316573.597049:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:0.0:1767316573.597056:0:2485:0:(recover.c:189:ptlrpc_wake_delayed()) @@@ waking (set 00000000e6e844f5): req@ffff88c5c57d4000 x1853163991566336/t0(0) o13->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316573.597079:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991566208:192.168.204.145@tcp:8:lfs.0 00000100:00100000:2.0:1767316573.597419:0:45106:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c351a600 going to sleep for 11 seconds 00010000:00080000:2.0:1767316573.609314:0:45106:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff88c5c57d4000 x1853163991566336/t0(0) o13->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:7/4 lens 224/368 e 0 to 0 dl 1767316589 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316573.609337:0:45106:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45106:1853163991566336:192.168.204.145@tcp:13:lfs.0 00020000:00000004:2.0:1767316573.609367:0:45106:0:(obd_class.h:1109:obd_statfs()) lustre-OST0001-osc-ffff88c5c68bd000: update ffff88c5d0094df8 cache blocks 3674/3683 objects 117632/118571 00000080:00200080:2.0:1767316573.610003:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610025:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610033:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610038:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610042:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610046:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610050:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610054:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610058:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610062:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610066:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610069:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610073:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610087:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610092:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610118:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610124:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610128:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610151:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610155:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610169:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610174:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610178:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610191:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610196:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610200:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610204:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610207:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610211:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610215:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610219:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610222:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610226:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610229:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610233:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610237:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610240:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610244:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610247:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610250:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610254:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610257:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610261:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610273:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610279:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610283:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610286:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610289:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610292:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610296:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610299:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610302:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610305:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610308:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610312:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610314:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610318:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610321:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610324:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610327:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610330:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610333:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610336:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610339:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610342:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610345:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610348:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610352:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610362:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610365:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610369:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610372:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610375:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610379:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610382:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610385:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610388:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610391:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610394:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610398:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610401:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610404:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610408:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610410:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610414:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610417:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610421:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610423:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610427:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610430:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610433:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610437:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610440:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610449:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610454:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610458:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610462:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610465:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610469:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610473:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610477:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610480:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610485:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610489:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610493:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610496:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610500:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610503:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610507:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610510:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610514:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610518:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610533:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610538:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610542:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610545:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610549:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610552:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610563:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610567:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610571:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610574:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610578:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610581:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200080:2.0:1767316573.610585:0:45106:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd217a8fc0 00020000:00000080:2.0:1767316573.610588:0:45106:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c6da8800 uarg=0000000000000000 00000080:00200008:2.0:1767316573.610612:0:45106:0:(file.c:424:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 100000 00010000:00010000:1.0:1767316573.959293:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316574.990444:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316576.008368:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1767316576.867008:0:45357:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg445-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 00010000:00010000:0.0:1767316577.033000:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316578.061088:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316578.439424:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991566464:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316578.439666:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991566720:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316578.439874:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1767316578.439882:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991566592:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316578.439914:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991566848:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316578.442669:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991566592:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316578.442703:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991566848:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316578.442952:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991566720:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316578.444310:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991566464:192.168.204.145@tcp:400:kworker.0 00000001:02000400:2.0:1767316578.885930:0:45512:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 00010000:00010000:0.0:1767316579.082800:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316580.106668:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:2.0:1767316580.725779:0:45550:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1767316580.725795:0:45550:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316580.725818:0:45550:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316580.725889:0:45550:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 45550 initiator: MDT0 00010000:00010000:2.0:1767316580.725900:0:45550:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 45550 initiator: MDT0 00010000:00010000:2.0:1767316580.725910:0:45550:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 45550 initiator: MDT0 00000100:00100000:2.0:1767316580.725944:0:45550:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45550:1853163991566976:192.168.204.145@tcp:101:lfs.0 00000100:00100000:2.0:1767316580.726039:0:45550:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426780 going to sleep for 11 seconds 00000100:00100000:1.0:1767316580.732974:0:45550:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45550:1853163991566976:192.168.204.145@tcp:101:lfs.0 00010000:00010000:1.0:1767316580.733001:0:45550:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.733014:0:45550:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.733025:0:45550:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.733036:0:45550:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.733047:0:45550:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.733065:0:45550:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.733072:0:45550:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316580.733335:0:45550:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d4000 x1853163991566976/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316596 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316580.733350:0:45550:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.733363:0:45550:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1767316580.733379:0:45550:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316580.733978:0:45550:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:0.0:1767316580.733991:0:45550:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d089ce 00000080:00010000:0.0:1767316580.734005:0:45550:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316580.734008:0:45550:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089ce from it ffffb10b00c87cb8 00010000:00010000:0.0:1767316580.734011:0:45550:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316580.734020:0:45550:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00000080:00200008:0.0:1767316580.735571:0:45550:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 0, flags 0, foreign 0 00000080:00200000:0.0:1767316580.735580:0:45550:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316580.735600:0:45550:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316580.735616:0:45550:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316580.735629:0:45550:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316580.735637:0:45550:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316580.735643:0:45550:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316580.735647:0:45550:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d089ce 00000080:00010000:0.0:1767316580.735652:0:45550:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089ce from it ffffb10b00c87cb8 00010000:00010000:0.0:1767316580.735656:0:45550:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316580.735664:0:45550:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316580.735792:0:45550:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316580.735801:0:45550:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316580.735812:0:45550:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316580.735925:0:45550:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316580.735934:0:45550:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316580.735938:0:45550:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316580.735941:0:45550:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d089ce 00000080:00010000:0.0:1767316580.735945:0:45550:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089ce from it ffffb10b00c87bb0 00010000:00010000:0.0:1767316580.735948:0:45550:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316580.735955:0:45550:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316580.735965:0:45550:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 44 00000080:00200008:0.0:1767316580.735973:0:45550:0:(file.c:1001:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 100000 00000080:00200080:0.0:1767316580.736126:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.736143:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00800000:00000004:0.0:1767316580.736150:0:45550:0:(obd_class.h:1083:obd_statfs()) lustre-MDT0000-mdc-ffff88c5c68bd000: age 1772, max_age 1778 00000100:00100000:0.0:1767316580.736343:0:45550:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45550:1853163991567104:192.168.204.145@tcp:41:lfs.0 00000100:00100000:0.0:1767316580.736586:0:45550:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c5754900 going to sleep for 11 seconds 00000100:00100000:0.0:1767316580.741397:0:45550:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45550:1853163991567104:192.168.204.145@tcp:41:lfs.0 00800000:00000004:0.0:1767316580.741427:0:45550:0:(obd_class.h:1109:obd_statfs()) lustre-MDT0000-mdc-ffff88c5c68bd000: update ffff88c5d0093520 cache blocks 17229/17271 objects 499700/500067 00000080:00200080:0.0:1767316580.741743:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741762:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741784:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741789:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741793:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741797:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741801:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741804:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741808:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741812:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741815:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741818:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741822:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741826:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741829:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741832:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741835:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741838:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741841:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741844:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741847:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741850:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741854:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741857:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741860:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741863:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741866:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741876:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741881:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741885:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741888:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741891:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741894:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741898:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741901:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741904:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741907:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741910:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741912:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741915:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741918:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741921:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741925:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741928:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741931:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741935:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741939:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741942:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741945:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741948:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741952:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741955:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741967:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741972:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741976:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741979:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741983:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741986:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741990:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741993:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.741997:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.741999:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742004:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742007:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742011:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742014:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742017:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742020:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742023:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742027:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742030:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742032:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742036:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742039:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742042:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742046:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742049:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742069:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742075:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742079:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742083:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742086:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742090:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742129:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742136:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742140:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742144:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742160:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742165:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742169:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742173:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742177:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742180:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742184:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742187:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742190:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742195:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742198:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742201:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742205:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742208:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742212:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742223:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742228:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742232:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742235:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742239:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742242:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742246:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742249:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742253:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742256:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742261:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742264:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742268:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742271:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742275:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742278:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742282:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742285:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742299:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742303:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742308:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742312:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742316:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742319:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742323:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00800000:00000080:0.0:1767316580.742333:0:45550:0:(lmv_obd.c:905:lmv_iocontrol()) lustre-clilmv-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.742338:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.742344:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00020000:00000004:0.0:1767316580.742351:0:45550:0:(obd_class.h:1083:obd_statfs()) lustre-OST0000-osc-ffff88c5c68bd000: age 1772, max_age 1778 00000100:00100000:0.0:1767316580.742391:0:45550:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45550:1853163991567232:192.168.204.145@tcp:13:lfs.0 00000100:00100000:0.0:1767316580.742491:0:45550:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c5754900 going to sleep for 11 seconds 00000100:00100000:0.0:1767316580.756441:0:45550:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45550:1853163991567232:192.168.204.145@tcp:13:lfs.0 00020000:00000004:0.0:1767316580.756473:0:45550:0:(obd_class.h:1109:obd_statfs()) lustre-OST0000-osc-ffff88c5c68bd000: update ffff88c5d0091c48 cache blocks 3678/3683 objects 117760/118175 00000080:00200080:0.0:1767316580.756697:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.756718:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00020000:00000004:0.0:1767316580.756734:0:45550:0:(obd_class.h:1083:obd_statfs()) lustre-OST0001-osc-ffff88c5c68bd000: age 1772, max_age 1778 00000100:00100000:0.0:1767316580.756770:0:45550:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45550:1853163991567360:192.168.204.145@tcp:13:lfs.0 00000100:00100000:0.0:1767316580.756843:0:45550:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c5754900 going to sleep for 11 seconds 00000100:00100000:0.0:1767316580.764740:0:45550:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45550:1853163991567360:192.168.204.145@tcp:13:lfs.0 00020000:00000004:0.0:1767316580.764771:0:45550:0:(obd_class.h:1109:obd_statfs()) lustre-OST0001-osc-ffff88c5c68bd000: update ffff88c5d0094df8 cache blocks 3674/3683 objects 117632/118571 00000080:00200080:0.0:1767316580.764850:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764866:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764873:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764876:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764880:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764883:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764887:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764890:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764894:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764910:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764914:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764917:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764921:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764924:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764928:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764931:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764935:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764938:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764942:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764946:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764950:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764953:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764956:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764960:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764964:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764968:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764971:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764975:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764979:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764982:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764986:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764989:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.764993:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.764997:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765021:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765026:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765031:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765034:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765038:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765042:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765045:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765048:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765062:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765067:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765070:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765074:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765078:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765082:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765085:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765089:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765123:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765129:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765133:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765136:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765140:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765143:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765685:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765694:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765700:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765710:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765714:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765718:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765731:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765735:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765741:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765745:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765749:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765752:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765756:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765759:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765763:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765766:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765769:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765773:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765776:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765779:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765783:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765786:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765790:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765793:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765796:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765799:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765803:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765806:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765815:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765827:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765831:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765835:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765839:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765842:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765846:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765849:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765853:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765856:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765859:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765863:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765866:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765869:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765873:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765876:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765880:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765883:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765886:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765889:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765893:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765896:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765900:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765903:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765906:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765914:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765919:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765923:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765927:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765930:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765934:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765937:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765941:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765944:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765947:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765951:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765954:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765957:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765961:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765964:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200080:0.0:1767316580.765967:0:45550:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) cmd=c00866a4 arg=7ffd39df5810 00020000:00000080:0.0:1767316580.765970:0:45550:0:(lov_obd.c:958:lov_iocontrol()) lustre-clilov-ffff88c5c68bd000: cmd=c00866a4 len=592 karg=ffff88c5c64d0c00 uarg=0000000000000000 00000080:00200008:0.0:1767316580.765983:0:45550:0:(file.c:424:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 100000 00010000:00010000:0.0:1767316580.824042:0:45569:0:(ldlm_resource.c:158:lru_size_store()) dropping all unused locks from namespace lustre-MDT0000-mdc-ffff88c5c68bd000 00010000:00010000:0.0:1767316580.824294:0:45569:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1767316580.824333:0:7227:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1767316580.824344:0:7227:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.824407:0:7227:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.824415:0:7227:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1767316580.824438:0:7227:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_01:5398f0ac-a698-45f7-862d-28236c50e3c9:7227:1853163991567488:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1767316580.824654:0:7227:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4ad9900 going to sleep for 11 seconds 00000100:00100000:1.0:1767316580.831667:0:7227:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ldlm_bl_01:5398f0ac-a698-45f7-862d-28236c50e3c9:7227:1853163991567488:192.168.204.145@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1767316580.831704:0:7227:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d089ce lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779517 expref: -99 pid: 45550 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.831755:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316580.831757:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316580.831760:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1767316580.870032:0:45576:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316580.870073:0:45576:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316580.870157:0:45576:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 45576 initiator: MDT0 00010000:00010000:2.0:1767316580.870168:0:45576:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00010000:00010000:2.0:1767316580.870178:0:45576:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00000100:00100000:2.0:1767316580.870203:0:45576:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991567616:192.168.204.145@tcp:101:lfs.0 00000100:00100000:2.0:1767316580.870306:0:45576:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426240 going to sleep for 11 seconds 00000100:00100000:2.0:1767316580.880832:0:45576:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991567616:192.168.204.145@tcp:101:lfs.0 00010000:00010000:2.0:1767316580.880856:0:45576:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316580.880869:0:45576:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316580.880880:0:45576:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316580.880890:0:45576:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316580.880903:0:45576:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316580.880924:0:45576:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316580.880930:0:45576:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316580.880941:0:45576:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d6a00 x1853163991567616/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316596 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316580.880955:0:45576:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316580.880968:0:45576:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1767316580.880975:0:45576:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1767316580.881000:0:45576:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:2.0:1767316580.881006:0:45576:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d089d5 00000080:00010000:2.0:1767316580.881013:0:45576:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316580.881016:0:45576:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089d5 from it ffffb10b01467bf8 00010000:00010000:2.0:1767316580.881019:0:45576:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316580.881026:0:45576:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316580.881061:0:45576:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316580.881083:0:45576:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=27 00000080:00200000:2.0:1767316580.881087:0:45576:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=0 00000002:00010000:2.0:1767316580.881123:0:45576:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d202.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1767316580.882602:0:45576:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 45576 initiator: MDT0 00010000:00010000:2.0:1767316580.882617:0:45576:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00010000:00010000:2.0:1767316580.882628:0:45576:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00000100:00100000:2.0:1767316580.882654:0:45576:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991567744:192.168.204.145@tcp:101:lfs.0 00000100:00100000:2.0:1767316580.882854:0:45576:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426240 going to sleep for 11 seconds 00000100:00100000:3.0:1767316580.888950:0:45576:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991567744:192.168.204.145@tcp:101:lfs.0 00010000:00010000:3.0:1767316580.888978:0:45576:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316580.889000:0:45576:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316580.889008:0:45576:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316580.889019:0:45576:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316580.889030:0:45576:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316580.889042:0:45576:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316580.889062:0:45576:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316580.889069:0:45576:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1767316580.889088:0:45576:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=10 disposition=b, status=0 req@ffff88c5c57d6a00 x1853163991567744/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316596 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1767316580.890239:0:45576:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316580.890259:0:45576:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1127 (0 0) 00000080:00200000:3.0:1767316580.890267:0:45576:0:(namei.c:955:ll_intent_lock()) intent lock 16 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:3.0:1767316580.890291:0:45576:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 11776 00000080:00200000:3.0:1767316580.890294:0:45576:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x467:0x0](ffff88c5c997ba88): rc = 0 00000080:00010000:3.0:1767316580.890300:0:45576:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) for lock 0x5295060e33d089dc 00000080:00010000:3.0:1767316580.890314:0:45576:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00010000:3.0:1767316580.890318:0:45576:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089dc from it ffffb10b01467ac8 00010000:00010000:3.0:1767316580.890322:0:45576:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316580.890330:0:45576:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1767316580.890364:0:45576:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 1 00000080:00200000:3.0:1767316580.890380:0:45576:0:(namei.c:1354:ll_atomic_open()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), file ffff88c5c68ac900, open_flags 18800, mode 0 opened 0 00000080:00200000:3.0:1767316580.890387:0:45576:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), intent=open 00000002:00010000:3.0:1767316580.890405:0:45576:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f202.replay-single,[0x0:0x0:0x0]) in obj [0x200000bd1:0x467:0x0], intent: open flags 0304001 00010000:00010000:3.0:1767316580.890450:0:45576:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 45576 initiator: MDT0 00010000:00010000:3.0:1767316580.890467:0:45576:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00000020:00100000:3.0:1767316580.890486:0:45576:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1767316580.890495:0:45576:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c5729c00 x1853163991568000/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/131968 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1767316580.890504:0:45576:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00000100:00100000:3.0:1767316580.891262:0:45576:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991568000:192.168.204.145@tcp:101:lfs.0 00000100:00100000:3.0:1767316580.891432:0:45576:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6084b40 going to sleep for 11 seconds 00000100:00100000:1.0:1767316580.898702:0:45576:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991568000:192.168.204.145@tcp:101:lfs.0 00010000:00010000:1.0:1767316580.898735:0:45576:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.898753:0:45576:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.898760:0:45576:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.898766:0:45576:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.898774:0:45576:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.898780:0:45576:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c4d1aa00 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:1.0:1767316580.898785:0:45576:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.898791:0:45576:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.898797:0:45576:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 5/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.898803:0:45576:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1767316580.898809:0:45576:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.898814:0:45576:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1aa00/0x5295060e33d089e3 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 45576 initiator: MDT0 00000002:00100000:1.0:1767316580.898832:0:45576:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=b, status=-20 req@ffff88c5c5729c00 x1853163991568000/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 776/608 e 0 to 0 dl 1767316596 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316580.898844:0:45576:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000bd1:0x467:0x0] suppgids 0 -1: rc -20 00000080:00200000:1.0:1767316580.898966:0:45576:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316580.898985:0:45576:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316580.898998:0:45576:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.899009:0:45576:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316580.899015:0:45576:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316580.899021:0:45576:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316580.899025:0:45576:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d089d5 00000080:00010000:1.0:1767316580.899031:0:45576:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089d5 from it ffffb10b01467bf8 00010000:00010000:1.0:1767316580.899034:0:45576:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316580.899042:0:45576:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316580.899069:0:45576:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316580.899077:0:45576:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d202.replay-single, flags=267 00000080:00200000:1.0:1767316580.899083:0:45576:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 44 00000080:00200008:1.0:1767316580.899089:0:45576:0:(file.c:1001:ll_file_open()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 304000 00000002:00010000:1.0:1767316580.899122:0:45576:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100240304001 00010000:00010000:1.0:1767316580.899413:0:45576:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 45576 initiator: MDT0 00010000:00010000:1.0:1767316580.899421:0:45576:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00000020:00100000:1.0:1767316580.899432:0:45576:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316580.899445:0:45576:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cf100 x1853163991568256/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/131968 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316580.899455:0:45576:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00000100:00100000:1.0:1767316580.899485:0:45576:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991568256:192.168.204.145@tcp:101:lfs.0 00000100:00100000:1.0:1767316580.899972:0:45576:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4ad9900 going to sleep for 11 seconds 00000100:00100000:0.0:1767316580.918660:0:45576:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991568256:192.168.204.145@tcp:101:lfs.0 00010000:00010000:0.0:1767316580.918682:0:45576:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 45576 initiator: MDT0 00010000:00010000:0.0:1767316580.918695:0:45576:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 45576 initiator: MDT0 00010000:00010000:0.0:1767316580.918718:0:45576:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 45576 initiator: MDT0 00010000:00010000:0.0:1767316580.918725:0:45576:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 45576 initiator: MDT0 00010000:00010000:0.0:1767316580.918733:0:45576:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 45576 initiator: MDT0 00010000:00010000:0.0:1767316580.918739:0:45576:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5d191b400 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:0.0:1767316580.918743:0:45576:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 45576 initiator: MDT0 00010000:00010000:0.0:1767316580.918750:0:45576:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 45576 initiator: MDT0 00010000:00010000:0.0:1767316580.918757:0:45576:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 45576 initiator: MDT0 00010000:00010000:0.0:1767316580.918763:0:45576:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1767316580.918769:0:45576:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 45576 initiator: MDT0 00010000:00010000:0.0:1767316580.918775:0:45576:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d089ea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 45576 initiator: MDT0 00000002:00100000:0.0:1767316580.918791:0:45576:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c65cf100 x1853163991568256/t51539607553(51539607553) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316596 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1767316580.918805:0:45576:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cf100 x1853163991568256/t51539607553(51539607553) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316596 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1767316580.918818:0:45576:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316580.918840:0:45576:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 11776 00000002:00100000:0.0:1767316580.918854:0:45576:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cf100 x1853163991568256/t51539607553(51539607553) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316596 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200080:0.0:1767316580.918943:0:45576:0:(dir.c:2017:ll_dir_ioctl()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88) cmd=c0086915 arg=1b963d0 00000100:00100000:0.0:1767316580.919003:0:45576:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991568384:192.168.204.145@tcp:34:lfs.0 00000100:00100000:0.0:1767316580.919092:0:45576:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c5754900 going to sleep for 11 seconds 00000100:00100000:0.0:1767316580.946954:0:45576:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991568384:192.168.204.145@tcp:34:lfs.0 00000080:00200008:0.0:1767316580.947956:0:45576:0:(file.c:424:ll_file_release()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 304000 00000002:00100000:0.0:1767316580.948027:0:45576:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cf100 x1853163991568256/t51539607553(51539607553) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316596 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1767316580.948041:0:45576:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1767316580.948056:0:45576:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6402300 x1853163991568640/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/736 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1767316580.948081:0:45576:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991568640:192.168.204.145@tcp:35:lfs.0 00000100:00100000:0.0:1767316580.950371:0:45576:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c5754900 going to sleep for 17 seconds 00000100:00100000:1.0:1767316580.957848:0:45576:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job lfs:5398f0ac-a698-45f7-862d-28236c50e3c9:45576:1853163991568640:192.168.204.145@tcp:35:lfs.0 00000002:00100000:1.0:1767316580.957879:0:45576:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cf100 x1853163991568256/t51539607553(51539607553) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 760/608 e 0 to 0 dl 1767316596 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'lfs.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1767316581.128331:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316582.160424:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316583.179599:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316583.574447:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991568768:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316583.574590:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991568896:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316583.574675:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991569024:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316583.574729:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991569152:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316583.582867:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991568768:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316583.587329:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991568896:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316583.588400:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991569024:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316583.588436:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991569152:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316584.200263:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316585.223340:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316586.248387:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316587.273309:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1767316587.301293:0:46138:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 203: resend can hit original request ========================================================== 20:16:26 (1767316586) 00000080:01200004:2.0:1767316587.698722:0:46159:0:(super25.c:139:lustre_fill_super()) VFS Op: sb ffff88c5ca304000 00000020:01000004:2.0:1767316587.698753:0:46159:0:(obd_mount.c:901:lmd_print()) mount data: 00000020:01000004:2.0:1767316587.698755:0:46159:0:(obd_mount.c:903:lmd_print()) profile: lustre-client 00000020:01000004:2.0:1767316587.698757:0:46159:0:(obd_mount.c:904:lmd_print()) device: 192.168.204.145@tcp:/lustre 00000020:01000004:2.0:1767316587.698758:0:46159:0:(obd_mount.c:907:lmd_print()) options: user_xattr,flock 00000080:01000004:2.0:1767316587.698760:0:46159:0:(super25.c:182:lustre_fill_super()) Mounting client lustre-client 00000020:01000004:2.0:1767316587.698781:0:46159:0:(obd_mount.c:392:lustre_start_mgc()) MGC192.168.204.145@tcp:Set MGC reconnect 1 10000000:01000000:2.0:1767316587.698785:0:46159:0:(mgc_request.c:1068:mgc_set_info_async()) InitRecov MGC192.168.204.145@tcp 1/d0:i0:r0:or0:FULL 00000080:00200000:2.0:1767316587.698793:0:46159:0:(llite_lib.c:1391:ll_fill_super()) VFS Op: cfg_instance lustre-client-ffff88c5ca304000 (sb ffff88c5ca304000) 00000080:01000000:2.0:1767316587.698870:0:46159:0:(llite_lib.c:1113:ll_options()) Parsing opts user_xattr,flock 00000080:00000004:2.0:1767316587.698872:0:46159:0:(llite_lib.c:1123:ll_options()) next opt=user_xattr 00000080:00000004:2.0:1767316587.698875:0:46159:0:(llite_lib.c:1123:ll_options()) next opt=flock 00000080:01000000:2.0:1767316587.698884:0:46159:0:(llite_lib.c:1425:ll_fill_super()) llite sb uuid: 4d658a6d-2c23-4e05-a7c2-999433a912b6 10000000:01000000:2.0:1767316587.699367:0:46159:0:(mgc_request.c:1952:mgc_process_config()) parse_log lustre-client from 0 10000000:01000000:2.0:1767316587.699371:0:46159:0:(mgc_request.c:329:config_log_add()) add config log lustre-client-ffff88c5ca304000 10000000:01000000:2.0:1767316587.699378:0:46159:0:(mgc_request.c:194:do_config_log_add()) do adding config log params-ffff88c5ca304000 10000000:01000000:2.0:1767316587.699382:0:46159:0:(mgc_request.c:68:mgc_name2resid()) log params to resid 0x736d61726170/0x3 (params) 10000000:01000000:2.0:1767316587.699387:0:46159:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-client-ffff88c5ca304000 10000000:01000000:2.0:1767316587.699389:0:46159:0:(mgc_request.c:68:mgc_name2resid()) log lustre-client to resid 0x65727473756c/0x0 (lustre) 10000000:01000000:2.0:1767316587.699393:0:46159:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-cliir-ffff88c5ca304000 10000000:01000000:2.0:1767316587.699395:0:46159:0:(mgc_request.c:68:mgc_name2resid()) log lustre-cliir to resid 0x65727473756c/0x2 (lustre) 10000000:01000000:2.0:1767316587.699398:0:46159:0:(mgc_request.c:1812:mgc_process_log()) Process log lustre-client-ffff88c5ca304000 from 1 10000000:01000000:2.0:1767316587.699406:0:46159:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1767316587.699443:0:46159:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316587.699454:0:46159:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316587.699466:0:46159:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1767316587.699491:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991569280:192.168.204.145@tcp:101:mount.lustre.0 00000100:00100000:2.0:1767316587.699587:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184be40 going to sleep for 11 seconds 00000100:00100000:3.0:1767316587.706424:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991569280:192.168.204.145@tcp:101:mount.lustre.0 00010000:00010000:3.0:1767316587.706458:0:46159:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316587.706472:0:46159:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1767316587.706569:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991569408:192.168.204.145@tcp:501:mount.lustre.0 00000100:00100000:3.0:1767316587.706736:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c2c55900 going to sleep for 11 seconds 00000100:00100000:0.0:1767316587.725249:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991569408:192.168.204.145@tcp:501:mount.lustre.0 00000100:00100000:0.0:1767316587.725317:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991569536:192.168.204.145@tcp:503:mount.lustre.0 00000100:00100000:0.0:1767316587.725402:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c863c0 going to sleep for 11 seconds 00000100:00100000:0.0:1767316587.742029:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991569536:192.168.204.145@tcp:503:mount.lustre.0 00000100:00100000:3.0:1767316587.744611:0:46163:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46163:1853163991569664:192.168.204.145@tcp:502:llog_process_th.0 00000100:00100000:3.0:1767316587.744818:0:46163:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c13709c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316587.751866:0:46163:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46163:1853163991569664:192.168.204.145@tcp:502:llog_process_th.0 00000020:01000000:1.0:1767316587.751919:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1767316587.751944:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.751947:0:46163:0:(obd_config.c:1477:class_process_config()) marker 4 (0x1) lustre-clilov lov setup 00000020:01000000:1.0:1767316587.751955:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf001, instance name: lustre-clilov-ffff88c5ca304000 00000020:00000080:1.0:1767316587.751958:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf001 00000020:00000080:1.0:1767316587.751972:0:46163:0:(genops.c:412:class_newdev()) Allocate new device lustre-clilov-ffff88c5ca304000 (ffff88c5c500b1b0) 00000020:00000080:1.0:1767316587.752002:0:46163:0:(obd_config.c:657:class_attach()) OBD: dev 6 attached type lov with refcount 2 00000020:01000000:1.0:1767316587.752009:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf003, instance name: lustre-clilov-ffff88c5ca304000 00000020:00000080:1.0:1767316587.752012:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf003 00000020:00000080:1.0:1767316587.752300:0:46163:0:(obd_config.c:761:class_setup()) finished setup of obd lustre-clilov-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:01000000:1.0:1767316587.752306:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.752310:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.752311:0:46163:0:(obd_config.c:1477:class_process_config()) marker 4 (0x2) lustre-clilov lov setup 00000020:01000000:1.0:1767316587.752320:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1767316587.752323:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.752325:0:46163:0:(obd_config.c:1477:class_process_config()) marker 5 (0x1) lustre-clilmv lmv setup 00000020:01000000:1.0:1767316587.752329:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf001, instance name: lustre-clilmv-ffff88c5ca304000 00000020:00000080:1.0:1767316587.752331:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf001 00000020:00000080:1.0:1767316587.752337:0:46163:0:(genops.c:412:class_newdev()) Allocate new device lustre-clilmv-ffff88c5ca304000 (ffff88c5c50098d8) 00000020:00000080:1.0:1767316587.752346:0:46163:0:(obd_config.c:657:class_attach()) OBD: dev 7 attached type lmv with refcount 2 00000020:01000000:1.0:1767316587.752364:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf003, instance name: lustre-clilmv-ffff88c5ca304000 00000020:00000080:1.0:1767316587.752366:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf003 00000020:00000080:1.0:1767316587.752537:0:46163:0:(obd_config.c:761:class_setup()) finished setup of obd lustre-clilmv-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:01000000:1.0:1767316587.752541:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.752544:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.752545:0:46163:0:(obd_config.c:1477:class_process_config()) marker 5 (0x2) lustre-clilmv lmv setup 00000020:01000000:1.0:1767316587.752548:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1767316587.752551:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.752552:0:46163:0:(obd_config.c:1477:class_process_config()) marker 6 (0x1) lustre-client mount opts 00000020:00000080:1.0:1767316587.752555:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf007 00000020:00000080:1.0:1767316587.752557:0:46163:0:(obd_config.c:1430:class_process_config()) mountopt: profile lustre-client osc lustre-clilov mdc lustre-clilmv 00000020:01000000:1.0:1767316587.752559:0:46163:0:(obd_config.c:1074:class_add_profile()) Add profile lustre-client 00000020:01000000:1.0:1767316587.752565:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.752567:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.752568:0:46163:0:(obd_config.c:1477:class_process_config()) marker 6 (0x2) lustre-client mount opts 00000020:01000000:1.0:1767316587.752570:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1767316587.752574:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.752575:0:46163:0:(obd_config.c:1477:class_process_config()) marker 7 (0x1) lustre-MDT0000 add mdc 00000020:00000080:1.0:1767316587.752577:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf005 00000020:00000080:1.0:1767316587.752582:0:46163:0:(obd_config.c:1401:class_process_config()) adding mapping from uuid 192.168.204.145@tcp to nid 0x20000c0a8cc91 (192.168.204.145@tcp) 00000020:01000000:1.0:1767316587.752606:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf001, instance name: lustre-MDT0000-mdc-ffff88c5ca304000 00000020:00000080:1.0:1767316587.752608:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf001 00000020:00000080:1.0:1767316587.752615:0:46163:0:(genops.c:412:class_newdev()) Allocate new device lustre-MDT0000-mdc-ffff88c5ca304000 (ffff88c5c500ca88) 00000020:00000080:1.0:1767316587.752623:0:46163:0:(obd_config.c:657:class_attach()) OBD: dev 8 attached type mdc with refcount 2 00000020:01000000:1.0:1767316587.752627:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf003, instance name: lustre-MDT0000-mdc-ffff88c5ca304000 00000020:00000080:1.0:1767316587.752636:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf003 00010000:00080000:1.0:1767316587.752699:0:46163:0:(ldlm_lib.c:102:import_set_conn()) imp ffff88c5c6a9e800@lustre-MDT0000-mdc-ffff88c5ca304000: add connection 192.168.204.145@tcp at head 00010000:00010000:1.0:1767316587.754272:0:46163:0:(ldlm_pool.c:924:ldlm_pool_init()) Lock pool ldlm-pool-lustre-MDT0000-mdc-ffff88c5ca304000-4 is initialized 00000040:01000000:1.0:1767316587.754479:0:46163:0:(llog_obd.c:195:llog_setup()) obd lustre-MDT0000-mdc-ffff88c5ca304000 ctxt 13 is initialized 00000020:00000080:1.0:1767316587.754523:0:46163:0:(obd_config.c:761:class_setup()) finished setup of obd lustre-MDT0000-mdc-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:01000000:1.0:1767316587.754534:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf014, instance name: lustre-clilmv-ffff88c5ca304000 00000020:00000080:1.0:1767316587.754538:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf014 00800000:01000000:1.0:1767316587.754559:0:46163:0:(lmv_obd.c:425:lmv_add_target()) Target uuid: lustre-MDT0000_UUID. index 0 00000020:01000000:1.0:1767316587.754598:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.754602:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.754604:0:46163:0:(obd_config.c:1477:class_process_config()) marker 7 (0x2) lustre-MDT0000 add mdc 00000020:01000000:1.0:1767316587.754607:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x5 00000020:01000000:1.0:1767316587.754609:0:46163:0:(obd_config.c:1797:class_config_llog_handler()) SKIP #9 00000020:01000000:1.0:1767316587.754611:0:46163:0:(obd_config.c:1823:class_config_llog_handler()) skipping 0x6 00000020:01000000:1.0:1767316587.754613:0:46163:0:(obd_config.c:1823:class_config_llog_handler()) skipping 0x6 00000020:01000000:1.0:1767316587.754614:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x6 mark_flg=0x6 00000020:00000080:1.0:1767316587.754617:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.754618:0:46163:0:(obd_config.c:1477:class_process_config()) marker 9 (0x6) lustre sys.timeout 00000020:01000000:1.0:1767316587.754621:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:01000004:1.0:1767316587.754626:0:46163:0:(obd_mount.c:939:lustre_check_exclusion()) Check exclusion lustre-OST0000 (0) in 0 of 192.168.204.145@tcp:/lustre 00000020:00000080:1.0:1767316587.754629:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.754630:0:46163:0:(obd_config.c:1477:class_process_config()) marker 13 (0x1) lustre-OST0000 add osc 00000020:00000080:1.0:1767316587.754633:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf005 00000020:00000080:1.0:1767316587.754637:0:46163:0:(obd_config.c:1401:class_process_config()) adding mapping from uuid 192.168.204.145@tcp to nid 0x20000c0a8cc91 (192.168.204.145@tcp) 00000020:01000000:1.0:1767316587.754661:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf001, instance name: lustre-OST0000-osc-ffff88c5ca304000 00000020:00000080:1.0:1767316587.754664:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf001 00000020:00000080:1.0:1767316587.754673:0:46163:0:(genops.c:412:class_newdev()) Allocate new device lustre-OST0000-osc-ffff88c5ca304000 (ffff88c5c5008000) 00000020:00000080:1.0:1767316587.754685:0:46163:0:(obd_config.c:657:class_attach()) OBD: dev 9 attached type osc with refcount 2 00000020:01000000:1.0:1767316587.754690:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf003, instance name: lustre-OST0000-osc-ffff88c5ca304000 00000020:00000080:1.0:1767316587.754693:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf003 00010000:00080000:1.0:1767316587.754759:0:46163:0:(ldlm_lib.c:102:import_set_conn()) imp ffff88c5c6a9a000@lustre-OST0000-osc-ffff88c5ca304000: add connection 192.168.204.145@tcp at head 00010000:00010000:1.0:1767316587.764782:0:46163:0:(ldlm_pool.c:924:ldlm_pool_init()) Lock pool ldlm-pool-lustre-OST0000-osc-ffff88c5ca304000-5 is initialized 00000020:00000080:1.0:1767316587.765110:0:46163:0:(obd_config.c:761:class_setup()) finished setup of obd lustre-OST0000-osc-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:01000000:1.0:1767316587.765128:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf00d, instance name: lustre-clilov-ffff88c5ca304000 00000020:00000080:1.0:1767316587.765134:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf00d 00020000:01000000:1.0:1767316587.765161:0:46163:0:(lov_obd.c:469:lov_add_target()) uuid:lustre-OST0000_UUID idx:0 gen:1 active:1 00020000:01000000:1.0:1767316587.765170:0:46163:0:(lov_obd.c:498:lov_add_target()) lustre-clilov-ffff88c5ca304000: Adding target lustre-OST0000_UUID 00020000:01000000:1.0:1767316587.765177:0:46163:0:(lov_obd.c:520:lov_add_target()) idx=0 ltd_gen=1 ld_tgt_count=1 00000020:01000000:1.0:1767316587.765190:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.765195:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.765197:0:46163:0:(obd_config.c:1477:class_process_config()) marker 13 (0x2) lustre-OST0000 add osc 00000020:01000000:1.0:1767316587.765202:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x5 00000020:01000000:1.0:1767316587.765205:0:46163:0:(obd_config.c:1797:class_config_llog_handler()) SKIP #16 00000020:01000000:1.0:1767316587.765207:0:46163:0:(obd_config.c:1823:class_config_llog_handler()) skipping 0x6 00000020:01000000:1.0:1767316587.765208:0:46163:0:(obd_config.c:1823:class_config_llog_handler()) skipping 0x6 00000020:01000000:1.0:1767316587.765210:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x6 mark_flg=0x6 00000020:00000080:1.0:1767316587.765212:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.765214:0:46163:0:(obd_config.c:1477:class_process_config()) marker 16 (0x6) lustre sys.timeout 00000020:01000000:1.0:1767316587.765216:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:01000004:1.0:1767316587.765221:0:46163:0:(obd_mount.c:939:lustre_check_exclusion()) Check exclusion lustre-OST0001 (1) in 0 of 192.168.204.145@tcp:/lustre 00000020:00000080:1.0:1767316587.765225:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.765227:0:46163:0:(obd_config.c:1477:class_process_config()) marker 19 (0x1) lustre-OST0001 add osc 00000020:00000080:1.0:1767316587.765230:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf005 00000020:00000080:1.0:1767316587.765236:0:46163:0:(obd_config.c:1401:class_process_config()) adding mapping from uuid 192.168.204.145@tcp to nid 0x20000c0a8cc91 (192.168.204.145@tcp) 00000020:01000000:1.0:1767316587.765265:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf001, instance name: lustre-OST0001-osc-ffff88c5ca304000 00000020:00000080:1.0:1767316587.765269:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf001 00000020:00000080:1.0:1767316587.765291:0:46163:0:(genops.c:412:class_newdev()) Allocate new device lustre-OST0001-osc-ffff88c5ca304000 (ffff88c5c5558000) 00000020:00000080:1.0:1767316587.765309:0:46163:0:(obd_config.c:657:class_attach()) OBD: dev 10 attached type osc with refcount 2 00000020:01000000:1.0:1767316587.765315:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf003, instance name: lustre-OST0001-osc-ffff88c5ca304000 00000020:00000080:1.0:1767316587.765318:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf003 00010000:00080000:1.0:1767316587.765720:0:46163:0:(ldlm_lib.c:102:import_set_conn()) imp ffff88c5c6a9c000@lustre-OST0001-osc-ffff88c5ca304000: add connection 192.168.204.145@tcp at head 00010000:00010000:1.0:1767316587.765908:0:46163:0:(ldlm_pool.c:924:ldlm_pool_init()) Lock pool ldlm-pool-lustre-OST0001-osc-ffff88c5ca304000-6 is initialized 00000020:00000080:1.0:1767316587.766154:0:46163:0:(obd_config.c:761:class_setup()) finished setup of obd lustre-OST0001-osc-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:01000000:1.0:1767316587.766163:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd cf00d, instance name: lustre-clilov-ffff88c5ca304000 00000020:00000080:1.0:1767316587.766170:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf00d 00020000:01000000:1.0:1767316587.766183:0:46163:0:(lov_obd.c:469:lov_add_target()) uuid:lustre-OST0001_UUID idx:1 gen:1 active:1 00020000:01000000:1.0:1767316587.766189:0:46163:0:(lov_obd.c:498:lov_add_target()) lustre-clilov-ffff88c5ca304000: Adding target lustre-OST0001_UUID 00020000:01000000:1.0:1767316587.766192:0:46163:0:(lov_obd.c:520:lov_add_target()) idx=1 ltd_gen=1 ld_tgt_count=2 00000020:01000000:1.0:1767316587.766199:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.766202:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.766204:0:46163:0:(obd_config.c:1477:class_process_config()) marker 19 (0x2) lustre-OST0001 add osc 00000020:01000000:1.0:1767316587.766207:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1767316587.766211:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.766213:0:46163:0:(obd_config.c:1477:class_process_config()) marker 22 (0x1) lustre sys.timeout 00000020:00000080:1.0:1767316587.766215:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf009 00000020:00000080:1.0:1767316587.766217:0:46163:0:(obd_config.c:1451:class_process_config()) changing lustre timeout from 20 to 20 00000020:01000000:1.0:1767316587.766219:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.766221:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.766222:0:46163:0:(obd_config.c:1477:class_process_config()) marker 22 (0x2) lustre sys.timeout 00000020:01000000:1.0:1767316587.766224:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1767316587.766226:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.766227:0:46163:0:(obd_config.c:1477:class_process_config()) marker 26 (0x1) lustre sys.jobid_var 00000020:00000080:1.0:1767316587.766229:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf00f 00000020:01000000:1.0:1767316587.766234:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.766237:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.766238:0:46163:0:(obd_config.c:1477:class_process_config()) marker 26 (0x2) lustre sys.jobid_var 00000020:01000000:1.0:1767316587.766240:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x5 00000020:01000000:1.0:1767316587.766243:0:46163:0:(obd_config.c:1797:class_config_llog_handler()) SKIP #27 00000020:01000000:1.0:1767316587.766244:0:46163:0:(obd_config.c:1823:class_config_llog_handler()) skipping 0x6 00000020:01000000:1.0:1767316587.766245:0:46163:0:(obd_config.c:1823:class_config_llog_handler()) skipping 0x6 00000020:01000000:1.0:1767316587.766247:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x6 mark_flg=0x6 00000020:00000080:1.0:1767316587.766249:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.766250:0:46163:0:(obd_config.c:1477:class_process_config()) marker 27 (0x6) lustre-clilov new lustre.pool_134 00000020:01000000:1.0:1767316587.766252:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x5 00000020:01000000:1.0:1767316587.766254:0:46163:0:(obd_config.c:1797:class_config_llog_handler()) SKIP #29 00000020:01000000:1.0:1767316587.766255:0:46163:0:(obd_config.c:1823:class_config_llog_handler()) skipping 0x6 00000020:01000000:1.0:1767316587.766256:0:46163:0:(obd_config.c:1823:class_config_llog_handler()) skipping 0x6 00000100:00100000:1.0:1767316587.766301:0:46163:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46163:1853163991569792:192.168.204.145@tcp:502:llog_process_th.0 00000100:00100000:1.0:1767316587.771152:0:46163:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c7d54cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316587.784729:0:46163:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46163:1853163991569792:192.168.204.145@tcp:502:llog_process_th.0 00000020:01000000:1.0:1767316587.784769:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x6 mark_flg=0x6 00000020:00000080:1.0:1767316587.784778:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.784780:0:46163:0:(obd_config.c:1477:class_process_config()) marker 29 (0x6) lustre-clilov add lustre.pool_134.lustre-OST0001_UUID 00000020:01000000:1.0:1767316587.784786:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1767316587.784790:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.784791:0:46163:0:(obd_config.c:1477:class_process_config()) marker 30 (0x1) lustre-clilov rem lustre.pool_134.lustre-OST0001_UUID 00000020:01000000:1.0:1767316587.784796:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd ce022, instance name: lustre-clilov-ffff88c5ca304000 00000020:00000080:1.0:1767316587.784799:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: ce022 00000020:01000000:1.0:1767316587.784809:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.784811:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.784812:0:46163:0:(obd_config.c:1477:class_process_config()) marker 30 (0x2) lustre-clilov rem lustre.pool_134.lustre-OST0001_UUID 00000020:01000000:1.0:1767316587.784815:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1767316587.784817:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.784818:0:46163:0:(obd_config.c:1477:class_process_config()) marker 32 (0x1) lustre-clilov del lustre.pool_134 00000020:01000000:1.0:1767316587.784821:0:46163:0:(obd_config.c:1903:class_config_llog_handler()) cmd ce023, instance name: lustre-clilov-ffff88c5ca304000 00000020:00000080:1.0:1767316587.784823:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: ce023 00000100:00100000:1.0:1767316587.784856:0:46163:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46163:1853163991569920:192.168.204.145@tcp:502:llog_process_th.0 00000100:00100000:1.0:1767316587.784947:0:46163:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c7d54cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316587.800566:0:46163:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46163:1853163991569920:192.168.204.145@tcp:502:llog_process_th.0 00000020:01000000:1.0:1767316587.800611:0:46163:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.800622:0:46163:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.800625:0:46163:0:(obd_config.c:1477:class_process_config()) marker 32 (0x2) lustre-clilov del lustre.pool_134 00000040:00080000:1.0:1767316587.800631:0:46163:0:(llog.c:854:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 55 count 54 00000020:01000000:0.0:1767316587.800781:0:46159:0:(obd_config.c:2043:class_config_parse_llog()) Processed log lustre-client gen 1-54 (rc=0) 10000000:01000000:0.0:1767316587.800800:0:46159:0:(mgc_request.c:1889:mgc_process_log()) MGC192.168.204.145@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1767316587.800820:0:46159:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316587.800832:0:46159:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 10000000:01000000:0.0:1767316587.800840:0:46159:0:(mgc_request.c:1812:mgc_process_log()) Process log lustre-cliir-ffff88c5ca304000 from 1 10000000:01000000:0.0:1767316587.800843:0:46159:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-cliir (res 0x65727473756c) 00010000:00010000:0.0:1767316587.800876:0:46159:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316587.800883:0:46159:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316587.800906:0:46159:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1767316587.800937:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570048:192.168.204.145@tcp:101:mount.lustre.0 00000100:00100000:0.0:1767316587.801024:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316587.815026:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570048:192.168.204.145@tcp:101:mount.lustre.0 00010000:00010000:0.0:1767316587.815055:0:46159:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316587.815068:0:46159:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1767316587.815918:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570176:192.168.204.145@tcp:256:mount.lustre.0 00000100:00100000:0.0:1767316587.816052:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316587.822744:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570176:192.168.204.145@tcp:256:mount.lustre.0 00000020:00000080:0.0:1767316587.822799:0:46159:0:(obd_config.c:1389:class_process_config()) processing cmd: cf00f 10000000:01000000:0.0:1767316587.823710:0:46159:0:(mgc_request.c:1889:mgc_process_log()) MGC192.168.204.145@tcp: configuration from log 'lustre-cliir' succeeded (0). 00010000:00010000:0.0:1767316587.823744:0:46159:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316587.823759:0:46159:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 10000000:01000000:0.0:1767316587.823770:0:46159:0:(mgc_request.c:1812:mgc_process_log()) Process log params-ffff88c5ca304000 from 1 10000000:01000000:0.0:1767316587.823773:0:46159:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:0.0:1767316587.823808:0:46159:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316587.823830:0:46159:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316587.823840:0:46159:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1767316587.823867:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570304:192.168.204.145@tcp:101:mount.lustre.0 00000100:00100000:0.0:1767316587.823953:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2000 going to sleep for 11 seconds 00000100:00100000:3.0:1767316587.836671:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570304:192.168.204.145@tcp:101:mount.lustre.0 00010000:00010000:3.0:1767316587.836699:0:46159:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316587.836711:0:46159:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1767316587.836814:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570432:192.168.204.145@tcp:501:mount.lustre.0 00000100:00100000:3.0:1767316587.836893:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c34f9f00 going to sleep for 11 seconds 00000100:00100000:3.0:1767316587.848030:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570432:192.168.204.145@tcp:501:mount.lustre.0 00000100:00100000:3.0:1767316587.848123:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570560:192.168.204.145@tcp:503:mount.lustre.0 00000100:00100000:3.0:1767316587.848214:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c34f9f00 going to sleep for 11 seconds 00000100:00100000:3.0:1767316587.854491:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:cf740287-d7f4-4a51-9260-7c6190205bc2:46159:1853163991570560:192.168.204.145@tcp:503:mount.lustre.0 00000100:00100000:3.0:1767316587.855670:0:46170:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46170:1853163991570688:192.168.204.145@tcp:502:llog_process_th.0 00000100:00100000:3.0:1767316587.855760:0:46170:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c34f9f00 going to sleep for 11 seconds 00000100:00100000:3.0:1767316587.859677:0:46170:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46170:1853163991570688:192.168.204.145@tcp:502:llog_process_th.0 00000020:01000000:3.0:1767316587.859716:0:46170:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:3.0:1767316587.859727:0:46170:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1767316587.859729:0:46170:0:(obd_config.c:1477:class_process_config()) marker 2 (0x1) general debug_raw_pointers 00000020:01000000:3.0:1767316587.859733:0:46170:0:(obd_config.c:1903:class_config_llog_handler()) cmd ce032, instance name: general-ffff88c5ca304000 00000020:00000080:3.0:1767316587.859736:0:46170:0:(obd_config.c:1389:class_process_config()) processing cmd: ce032 00000020:01000000:3.0:1767316587.859742:0:46170:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:3.0:1767316587.859744:0:46170:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1767316587.859745:0:46170:0:(obd_config.c:1477:class_process_config()) marker 2 (0x2) general debug_raw_pointers 00000020:01000000:3.0:1767316587.859746:0:46170:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:3.0:1767316587.859748:0:46170:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1767316587.859749:0:46170:0:(obd_config.c:1477:class_process_config()) marker 3 (0x1) general lod.*.mdt_hash 00000020:01000000:3.0:1767316587.859750:0:46170:0:(obd_config.c:1903:class_config_llog_handler()) cmd ce032, instance name: general-ffff88c5ca304000 00000020:00000080:3.0:1767316587.859751:0:46170:0:(obd_config.c:1389:class_process_config()) processing cmd: ce032 00000100:00100000:3.0:1767316587.859829:0:46170:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46170:1853163991570816:192.168.204.145@tcp:502:llog_process_th.0 00000100:00100000:3.0:1767316587.859908:0:46170:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c351a600 going to sleep for 11 seconds 00000100:00100000:1.0:1767316587.879971:0:46170:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:cf740287-d7f4-4a51-9260-7c6190205bc2:46170:1853163991570816:192.168.204.145@tcp:502:llog_process_th.0 00000020:01000000:1.0:1767316587.880022:0:46170:0:(obd_config.c:1788:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1767316587.880040:0:46170:0:(obd_config.c:1389:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1767316587.880043:0:46170:0:(obd_config.c:1477:class_process_config()) marker 3 (0x2) general lod.*.mdt_hash 00000040:00080000:1.0:1767316587.880051:0:46170:0:(llog.c:854:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 7 count 7 00000020:01000000:0.0:1767316587.880372:0:46159:0:(obd_config.c:2043:class_config_parse_llog()) Processed log params gen 1-6 (rc=0) 10000000:01000000:0.0:1767316587.880392:0:46159:0:(mgc_request.c:1889:mgc_process_log()) MGC192.168.204.145@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:0.0:1767316587.880447:0:46159:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316587.880458:0:46159:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00000080:01000000:0.0:1767316587.880479:0:46159:0:(llite_lib.c:1495:ll_fill_super()) Found profile lustre-client: mdc=lustre-clilmv osc=lustre-clilov 00000020:00000080:0.0:1767316587.880506:0:46159:0:(genops.c:1402:class_connect()) connect: client 4d658a6d-2c23-4e05-a7c2-999433a912b6, cookie 0x5295060e33d08a29 00800000:01000000:0.0:1767316587.880529:0:46159:0:(lmv_obd.c:498:lmv_check_connect()) Time to connect 4d658a6d-2c23-4e05-a7c2-999433a912b6 to lustre-clilmv-ffff88c5ca304000 00800000:01000000:0.0:1767316587.880538:0:46159:0:(lmv_obd.c:338:lmv_connect_mdc()) connect to lustre-MDT0000-mdc-ffff88c5ca304000(4d658a6d-2c23-4e05-a7c2-999433a912b6) - lustre-MDT0000_UUID, 4d658a6d-2c23-4e05-a7c2-999433a912b6 00000020:00000080:0.0:1767316587.880556:0:46159:0:(genops.c:1402:class_connect()) connect: client 4d658a6d-2c23-4e05-a7c2-999433a912b6, cookie 0x5295060e33d08a30 00000100:00080000:0.0:1767316587.880565:0:46159:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5ca304000: connect to NID 192.168.204.145@tcp last attempt 0 00000100:00080000:0.0:1767316587.880574:0:46159:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5ca304000: import ffff88c5c6a9e800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767316587.880586:0:46159:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 134217728, lost: 0. 00000100:00100000:0.0:1767316587.880612:0:46159:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403b80 x1853163991570944/t0(0) o38->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767316587.880652:0:46159:0:(pinger.c:371:ptlrpc_pinger_add_import()) adding pingable import 4d658a6d-2c23-4e05-a7c2-999433a912b6->lustre-MDT0000_UUID 00800000:01000000:0.0:1767316587.880747:0:46159:0:(lmv_obd.c:395:lmv_connect_mdc()) Connected to lustre-MDT0000-mdc-ffff88c5ca304000(4d658a6d-2c23-4e05-a7c2-999433a912b6) successfully (5) 00800000:00000004:0.0:1767316587.880751:0:46159:0:(obd_class.h:1049:obd_statfs_async()) lustre-MDT0000-mdc-ffff88c5ca304000: age 786, max_age 0 00000080:00000004:0.0:1767316587.880782:0:46159:0:(obd_class.h:1083:obd_statfs()) lustre-clilmv-ffff88c5ca304000: age 786, max_age 1786 00800000:00000004:0.0:1767316587.880789:0:46159:0:(obd_class.h:1083:obd_statfs()) lustre-MDT0000-mdc-ffff88c5ca304000: age 786, max_age 1786 00000100:00080000:0.0:1767316587.880807:0:46159:0:(client.c:1829:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff88c5c6401c00 x1853163991571200/t0(0) o41->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316587.880817:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a29c0 going to sleep for 0 seconds 00000100:00080000:0.0:1767316587.880837:0:2487:0:(client.c:1829:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff88c5c6403100 x1853163991571072/t0(0) o41->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316587.880965:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991570944:192.168.204.145@tcp:38:mount.lustre.0 00000100:00100000:2.0:1767316587.881274:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991571328:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316587.881313:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991571584:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316587.881340:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316587.881346:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991571456:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316587.886531:0:2488:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5c68bd000: committing for last_committed 51539607554 gen 2 00000100:00100000:3.0:1767316587.886558:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991571328:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316587.888804:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991571456:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316587.888823:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991571584:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316587.888940:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5ca304000: committing for last_committed 0 gen 1 00000100:00080000:1.0:1767316587.888950:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff88c5ca304000: connect to target with instance 12 00000100:00080000:1.0:1767316587.888960:0:2485:0:(import.c:951:ptlrpc_connect_set_flags()) lustre-MDT0000-mdc-ffff88c5ca304000: Resetting ns_connect_flags to server flags: 0xae7a5e7be344d3b8 00000100:00080000:1.0:1767316587.888972:0:2485:0:(import.c:1204:ptlrpc_connect_interpret()) connected to replayable target: lustre-MDT0000_UUID 00000100:00080000:1.0:1767316587.888977:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6a9e800 lustre-MDT0000_UUID: changing import state from CONNECTING to FULL 00000100:00080000:1.0:1767316587.889678:0:2485:0:(recover.c:189:ptlrpc_wake_delayed()) @@@ waking (set 0000000006bd8474): req@ffff88c5c6401c00 x1853163991571200/t0(0) o41->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316587.889705:0:2485:0:(recover.c:189:ptlrpc_wake_delayed()) @@@ waking (set 00000000e3b59a3f): req@ffff88c5c6403100 x1853163991571072/t0(0) o41->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 224/368 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/200/ffffffff rc 0/-1 job:'mount.lustre.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316587.889722:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991570944:192.168.204.145@tcp:38:mount.lustre.0 00000100:00100000:0.0:1767316587.890022:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a29c0 going to sleep for 11 seconds 00000100:00100000:0.0:1767316587.899891:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:4d658a6d-2c23-4e05-a7c2-999433a912b6:46159:1853163991571200:192.168.204.145@tcp:41:mount.lustre.0 00800000:00000004:0.0:1767316587.899934:0:46159:0:(obd_class.h:1109:obd_statfs()) lustre-MDT0000-mdc-ffff88c5ca304000: update ffff88c5c500cdf8 cache blocks 17229/17271 objects 499700/500067 00000080:00000004:0.0:1767316587.899942:0:46159:0:(obd_class.h:1109:obd_statfs()) lustre-clilmv-ffff88c5ca304000: update ffff88c5c5009c48 cache blocks 17229/17271 objects 499700/500067 00000100:00100000:0.0:1767316587.899989:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:4d658a6d-2c23-4e05-a7c2-999433a912b6:46159:1853163991571712:192.168.204.145@tcp:40:mount.lustre.0 00000100:00100000:0.0:1767316587.900060:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a23c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316587.900177:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4d658a6d-2c23-4e05-a7c2-999433a912b6:0:1853163991571072:192.168.204.145@tcp:41:mount.lustre.0 00000100:00100000:0.0:1767316587.903731:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job mount.lustre:4d658a6d-2c23-4e05-a7c2-999433a912b6:46159:1853163991571712:192.168.204.145@tcp:40:mount.lustre.0 00000080:00000004:0.0:1767316587.903761:0:46159:0:(llite_lib.c:511:client_common_fill_super()) rootfid [0x200000007:0x1:0x0] 00000080:00100000:0.0:1767316587.903777:0:46159:0:(llite_lib.c:674:client_common_fill_super()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34668544 ocd_grant: 3407872 00020000:01000000:0.0:1767316587.903781:0:46159:0:(lov_obd.c:186:lov_connect()) connect #0 00000020:00000080:0.0:1767316587.903794:0:46159:0:(genops.c:1402:class_connect()) connect: client 4d658a6d-2c23-4e05-a7c2-999433a912b6, cookie 0x5295060e33d08a37 00000020:00000080:0.0:1767316587.903811:0:46159:0:(genops.c:1402:class_connect()) connect: client LOV_OSC_UUID, cookie 0x5295060e33d08a3e 00000100:00080000:0.0:1767316587.903818:0:46159:0:(import.c:509:import_select_connection()) lustre-OST0000-osc-ffff88c5ca304000: connect to NID 192.168.204.145@tcp last attempt 0 00000100:00080000:0.0:1767316587.903825:0:46159:0:(import.c:623:import_select_connection()) lustre-OST0000-osc-ffff88c5ca304000: import ffff88c5c6a9a000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767316587.903851:0:46159:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34668544 ocd_grant: 134217728, lost: 0. 00000100:00100000:0.0:1767316587.903878:0:46159:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401c00 x1853163991571840/t0(0) o8->lustre-OST0000-osc-ffff88c5ca304000@192.168.204.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767316587.903910:0:46159:0:(pinger.c:371:ptlrpc_pinger_add_import()) adding pingable import 4d658a6d-2c23-4e05-a7c2-999433a912b6->lustre-OST0000_UUID 00020000:01000000:0.0:1767316587.903938:0:46159:0:(lov_obd.c:159:lov_connect_osc()) Connected tgt idx 0 lustre-OST0000_UUID (lustre-OST0000-osc-ffff88c5ca304000) active 00000080:00000004:0.0:1767316587.903958:0:46159:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0x0 -> 0xa0465af2e3640038 00000020:00000080:0.0:1767316587.903968:0:46159:0:(genops.c:1402:class_connect()) connect: client LOV_OSC_UUID, cookie 0x5295060e33d08a45 00000100:00080000:0.0:1767316587.903974:0:46159:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff88c5ca304000: connect to NID 192.168.204.145@tcp last attempt 0 00000100:00080000:0.0:1767316587.903978:0:46159:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff88c5ca304000: import ffff88c5c6a9c000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767316587.903984:0:46159:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34668544 ocd_grant: 134217728, lost: 0. 00000100:00100000:0.0:1767316587.903994:0:46159:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402d80 x1853163991571968/t0(0) o8->lustre-OST0001-osc-ffff88c5ca304000@192.168.204.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767316587.904002:0:46159:0:(pinger.c:371:ptlrpc_pinger_add_import()) adding pingable import 4d658a6d-2c23-4e05-a7c2-999433a912b6->lustre-OST0001_UUID 00020000:01000000:0.0:1767316587.904008:0:46159:0:(lov_obd.c:159:lov_connect_osc()) Connected tgt idx 1 lustre-OST0001_UUID (lustre-OST0001-osc-ffff88c5ca304000) active 00000080:00000004:0.0:1767316587.904015:0:46159:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0x0 -> 0xa0465af2e3640038 00000100:00100000:0.0:1767316587.904048:0:46159:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job mount.lustre:4d658a6d-2c23-4e05-a7c2-999433a912b6:46159:1853163991572096:192.168.204.145@tcp:33:mount.lustre.0 00000100:00100000:0.0:1767316587.904209:0:46159:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a23c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316587.904296:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991571840:192.168.204.145@tcp:8:mount.lustre.0 00000100:00100000:1.0:1767316587.904341:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991571968:192.168.204.145@tcp:8:mount.lustre.0 00000100:00100000:1.0:1767316587.908767:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-ffff88c5ca304000: committing for last_committed 0 gen 1 00000100:00080000:1.0:1767316587.908784:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) lustre-OST0000-osc-ffff88c5ca304000: connect to target with instance 4 00000100:00080000:1.0:1767316587.908795:0:2485:0:(import.c:951:ptlrpc_connect_set_flags()) lustre-OST0000-osc-ffff88c5ca304000: Resetting ns_connect_flags to server flags: 0xa0425af2e3440038 00000080:00000004:1.0:1767316587.908810:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0465af2e3640038 -> 0xa0425af2e3440038 00000080:00080000:1.0:1767316587.908817:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:1.0:1767316587.908826:0:2485:0:(import.c:1204:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0000_UUID 00000100:00080000:1.0:1767316587.908845:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6a9a000 lustre-OST0000_UUID: changing import state from CONNECTING to FULL 00020000:01000000:1.0:1767316587.908851:0:2485:0:(lov_obd.c:389:lov_set_osc_active()) Marking OSC lustre-OST0000_UUID active 00000080:00000004:1.0:1767316587.908854:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:1.0:1767316587.908856:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00100000:1.0:1767316587.908867:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991571840:192.168.204.145@tcp:8:mount.lustre.0 00000100:00100000:1.0:1767316587.908882:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-ffff88c5ca304000: committing for last_committed 0 gen 1 00000100:00080000:1.0:1767316587.908887:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff88c5ca304000: connect to target with instance 2 00000100:00080000:1.0:1767316587.908892:0:2485:0:(import.c:951:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff88c5ca304000: Resetting ns_connect_flags to server flags: 0xa0425af2e3440038 00000080:00000004:1.0:1767316587.908896:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:1.0:1767316587.908898:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:1.0:1767316587.908900:0:2485:0:(import.c:1204:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:1.0:1767316587.908902:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6a9c000 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00020000:01000000:1.0:1767316587.908905:0:2485:0:(lov_obd.c:389:lov_set_osc_active()) Marking OSC lustre-OST0001_UUID active 00000080:00000004:1.0:1767316587.908907:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:1.0:1767316587.908909:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00100000:1.0:1767316587.908913:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991571968:192.168.204.145@tcp:8:mount.lustre.0 00000100:00100000:0.0:1767316587.910380:0:46159:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job mount.lustre:4d658a6d-2c23-4e05-a7c2-999433a912b6:46159:1853163991572096:192.168.204.145@tcp:33:mount.lustre.0 00000080:00200000:0.0:1767316587.910500:0:46159:0:(llite_lib.c:3217:ll_read_inode2()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248) 00000080:00200000:0.0:1767316587.910509:0:46159:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:02000400:0.0:1767316587.910588:0:46159:0:(llite_lib.c:1533:ll_fill_super()) Mounted lustre-client 00000080:00000004:0.0:1767316587.912042:0:46159:0:(super25.c:202:lustre_fill_super()) 192.168.204.145@tcp:/lustre: Mount complete 00010000:00010000:1.0:1767316588.297405:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:1.0:1767316589.132990:0:46197:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316589.133017:0:46197:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316589.133034:0:46197:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316589.133049:0:46197:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316589.133057:0:46197:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316589.133064:0:46197:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316589.133068:0:46197:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d089d5 00000080:00010000:1.0:1767316589.133081:0:46197:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089d5 from it ffffb10b01e57b58 00010000:00010000:1.0:1767316589.133085:0:46197:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316589.133111:0:46197:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316589.133129:0:46197:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316589.133143:0:46197:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=f203.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:1.0:1767316589.133147:0:46197:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f203.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:1.0:1767316589.133160:0:46197:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f203.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316589.133206:0:46197:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46197 initiator: MDT0 00010000:00010000:1.0:1767316589.133214:0:46197:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46197 initiator: MDT0 00010000:00010000:1.0:1767316589.133223:0:46197:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46197 initiator: MDT0 00000100:00100000:1.0:1767316589.133249:0:46197:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job stat:5398f0ac-a698-45f7-862d-28236c50e3c9:46197:1853163991572224:192.168.204.145@tcp:101:stat.0 00000100:00100000:1.0:1767316589.134166:0:46197:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c7d54600 going to sleep for 11 seconds 00010000:00010000:1.0:1767316589.319404:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1767316589.657443:0:46199:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248),name=/ 00000002:00010000:2.0:1767316589.657479:0:46199:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316589.657629:0:46199:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46199 initiator: MDT0 00010000:00010000:2.0:1767316589.659470:0:46199:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00010000:00010000:2.0:1767316589.659504:0:46199:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00000100:00100000:2.0:1767316589.659537:0:46199:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991572352:192.168.204.145@tcp:101:lfs.0 00000100:00100000:2.0:1767316589.661136:0:46199:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d184b900 going to sleep for 11 seconds 00000100:00100000:3.0:1767316589.672603:0:46199:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991572352:192.168.204.145@tcp:101:lfs.0 00010000:00010000:3.0:1767316589.672631:0:46199:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316589.672653:0:46199:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316589.672947:0:46199:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316589.672966:0:46199:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316589.672979:0:46199:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316589.672986:0:46199:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316589.672993:0:46199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1767316589.673005:0:46199:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d5c00 x1853163991572352/t0(0) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316605 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1767316589.673037:0:46199:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316589.673051:0:46199:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c9f99000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1767316589.673058:0:46199:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1767316589.673085:0:46199:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:3.0:1767316589.673090:0:46199:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) for lock 0x5295060e33d08a53 00000080:00010000:3.0:1767316589.673161:0:46199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) 00000080:00010000:3.0:1767316589.673165:0:46199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a53 from it ffffb10b01c43b58 00010000:00010000:3.0:1767316589.673168:0:46199:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316589.673176:0:46199:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1767316589.673201:0:46199:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248), inode mode 41ed mask 1 00000080:00200000:3.0:1767316589.673223:0:46199:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=f203.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c997b248), flags=8 00000080:00200000:3.0:1767316589.673227:0:46199:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f203.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c997b248), intent=getattr 00000002:00010000:3.0:1767316589.673251:0:46199:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f203.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1767316589.673293:0:46199:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.675061:0:46199:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.675077:0:46199:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00000100:00100000:3.0:1767316589.675156:0:46199:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991572480:192.168.204.145@tcp:101:lfs.0 00000100:00100000:3.0:1767316589.678272:0:46199:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c34f9f00 going to sleep for 11 seconds 00000100:00100000:3.0:1767316589.694082:0:46199:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991572480:192.168.204.145@tcp:101:lfs.0 00010000:00010000:3.0:1767316589.694128:0:46199:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.694142:0:46199:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.694149:0:46199:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.694156:0:46199:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.694164:0:46199:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.694171:0:46199:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c4d1a400 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:3.0:1767316589.694176:0:46199:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.694183:0:46199:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x206400000000 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.694190:0:46199:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x20e400000000 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.694197:0:46199:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1767316589.694203:0:46199:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46199 initiator: MDT0 00010000:00010000:3.0:1767316589.694210:0:46199:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1a400/0x5295060e33d08a5a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46199 initiator: MDT0 00000002:00100000:3.0:1767316589.694938:0:46199:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@ffff88c5c5729f80 x1853163991572480/t0(0) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316605 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:3.0:1767316589.694956:0:46199:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00010000:00010000:0.0:1767316589.695054:0:46199:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.695080:0:46199:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000080:00010000:0.0:1767316589.695088:0:46199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a53 from it ffffb10b01c43940 00010000:00010000:0.0:1767316589.695125:0:46199:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.695134:0:46199:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316589.695221:0:46199:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248),name=/ 00000002:00010000:0.0:1767316589.695492:0:46199:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316589.695501:0:46199:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.695510:0:46199:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316589.695878:0:46199:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316589.695885:0:46199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) 00000080:00010000:0.0:1767316589.695898:0:46199:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) for lock 0x5295060e33d08a53 00000080:00010000:0.0:1767316589.695903:0:46199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a53 from it ffffb10b01c43bf8 00010000:00010000:0.0:1767316589.695906:0:46199:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.695912:0:46199:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316589.695937:0:46199:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248), inode mode 41ed mask 1 00000080:00200000:0.0:1767316589.695950:0:46199:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f203.replay-single, flags=777 00000080:00200000:0.0:1767316589.695955:0:46199:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248),name=/ 00000002:00010000:0.0:1767316589.695963:0:46199:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316589.695970:0:46199:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.695977:0:46199:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316589.695983:0:46199:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316589.695987:0:46199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) 00000080:00010000:0.0:1767316589.695989:0:46199:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) for lock 0x5295060e33d08a53 00000080:00010000:0.0:1767316589.695993:0:46199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a53 from it ffffb10b01c43b38 00010000:00010000:0.0:1767316589.695995:0:46199:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.696002:0:46199:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316589.696009:0:46199:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248), inode mode 41ed mask 3 00000080:00200000:0.0:1767316589.696014:0:46199:0:(namei.c:1354:ll_atomic_open()) VFS Op:name=f203.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c997b248), file ffff88c5c68eaf00, open_flags a141, mode 81b6 opened 0 00000080:00200000:0.0:1767316589.696024:0:46199:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f203.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c997b248), intent=open|creat 00000100:00100000:0.0:1767316589.696082:0:46199:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991572608:192.168.204.145@tcp:700:lfs.0 00000100:00100000:0.0:1767316589.697073:0:46199:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2a80 going to sleep for 11 seconds 00000100:00100000:0.0:1767316589.719279:0:46199:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991572608:192.168.204.145@tcp:700:lfs.0 00000002:00010000:0.0:1767316589.719348:0:46199:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f203.replay-single,[0x2000059f1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0120502 00010000:00010000:0.0:1767316589.719371:0:46199:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.719430:0:46199:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.719437:0:46199:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1767316589.719441:0:46199:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c606a600/0x5295060e33d08a53 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67795b8 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.719484:0:46199:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.719490:0:46199:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00000020:00100000:0.0:1767316589.719500:0:46199:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5ca304000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1767316589.719509:0:46199:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6400000 x1853163991572864/t0(0) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 536/66432 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1767316589.719532:0:46199:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00000100:00100000:0.0:1767316589.719554:0:46199:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991572864:192.168.204.145@tcp:101:lfs.0 00000100:00100000:0.0:1767316589.719734:0:46199:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2a80 going to sleep for 11 seconds 00000100:00100000:2.0:1767316589.727004:0:2481:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.145@tcp (source: 12345-192.168.204.145@tcp) 00000100:00100000:3.0:1767316589.727154:0:7218:0:(service.c:153:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1767316589.727176:0:7218:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1853164044876032 00000100:00100000:3.0:1767316589.727185:0:7218:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1853164044876032 00000100:00100000:3.0:1767316589.727247:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.145@tcp, seq: 1413 00000100:00100000:3.0:1767316589.727531:0:7218:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff88c5c6583850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:5398f0ac-a698-45f7-862d-28236c50e3c9+5:52162:x1853164044876032:12345-192.168.204.145@tcp:104: 00010000:00010000:3.0:1767316589.727548:0:7218:0:(ldlm_lockd.c:2505:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316589.727919:0:7218:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1767316589.727927:0:7218:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff88c5c6583850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:5398f0ac-a698-45f7-862d-28236c50e3c9+5:52162:x1853164044876032:12345-192.168.204.145@tcp:104: Request processed in 405us (933us total) trans 0 rc 0/0 00000100:00100000:3.0:1767316589.727935:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.145@tcp, seq: 1413 00010000:00010000:0.0:1767316589.732969:0:7704:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1767316589.732984:0:7704:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x400000000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.732998:0:7704:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c606b800 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:0.0:1767316589.733020:0:7704:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.733028:0:7704:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x400400000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.733038:0:7704:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x408400000000 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316589.741906:0:7704:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316589.741924:0:7704:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:0.0:1767316589.741998:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991572992:192.168.204.145@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1767316589.742181:0:7704:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316589.742191:0:7704:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d089d5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x13 rrc: 3 type: IBT gid 0 flags: 0x4c09400000020 nid: local remote: 0x60377922a6779525 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316589.742229:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316589.742232:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316589.742246:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316589.762127:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991572992:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1767316589.782075:0:46199:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991572864:192.168.204.145@tcp:101:lfs.0 00010000:00010000:0.0:1767316589.782156:0:46199:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.782189:0:46199:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 4/0,1 mode: --/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.782198:0:46199:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 4/0,1 mode: --/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.782209:0:46199:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 4/0,1 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.782221:0:46199:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 4/0,1 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.782234:0:46199:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 5/1,1 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.782356:0:46199:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 5/1,1 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.782363:0:46199:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 4/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316589.782376:0:46199:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=3 disposition=8000037, status=0 req@ffff88c5c6400000 x1853163991572864/t51539607555(51539607555) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 536/608 e 0 to 0 dl 1767316605 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1767316589.782388:0:46199:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6400000 x1853163991572864/t51539607555(51539607555) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 536/608 e 0 to 0 dl 1767316605 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316589.782401:0:46199:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 3/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.782411:0:46199:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c9f99000 type 13 mode 4 res 8589957617/1 (0 0) 00000080:00200000:0.0:1767316589.782417:0:46199:0:(namei.c:955:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1767316589.782429:0:46199:0:(namei.c:1569:ll_create_it()) VFS Op:name=f203.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c997b248), intent=open|creat 00000080:00200000:0.0:1767316589.782459:0:46199:0:(llite_lib.c:3217:ll_read_inode2()) VFS Op:inode=[0x2000059f1:0x1:0x0](ffff88c5c9bb42c8) 00000080:00200000:0.0:1767316589.782465:0:46199:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x2000059f1:0x1:0x0], updating i_size 0 00000080:00200000:0.0:1767316589.782753:0:46199:0:(file.c:6809:ll_layout_conf()) [0x2000059f1:0x1:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:0.0:1767316589.782759:0:46199:0:(namei.c:1535:ll_create_node()) setting l_ast_data to inode [0x2000059f1:0x1:0x0](ffff88c5c9bb42c8) 00000080:00010000:0.0:1767316589.782762:0:46199:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x2000059f1:0x1:0x0](ffff88c5c9bb42c8) for lock 0x5295060e33d08a61 00000080:00010000:0.0:1767316589.782773:0:46199:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000059f1:0x1:0x0](ffff88c5c9bb42c8) 00000080:00200008:0.0:1767316589.782790:0:46199:0:(file.c:1001:ll_file_open()) START file f203.replay-single:[0x2000059f1:0x1:0x0](ffff88c5c9bb42c8), flags 120501 00000002:00100000:0.0:1767316589.782811:0:46199:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6400000 x1853163991572864/t51539607555(51539607555) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 536/608 e 0 to 0 dl 1767316605 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1767316589.782859:0:46199:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a61 from it ffff88c5ca32f600 00010000:00010000:0.0:1767316589.782863:0:46199:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 3/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.782871:0:46199:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 2/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316589.782903:0:46199:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x2000059f1:0x1:0x0](ffff88c5c9bb42c8), inode mode 81a4 mask 40 00000080:00200000:0.0:1767316589.782960:0:46199:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x2000059f1:0x1:0x0](ffff88c5c9bb42c8), inode mode 81a4 mask 2 00000080:00200000:0.0:1767316589.782970:0:46199:0:(xattr.c:406:ll_xattr_set()) VFS Op:inode=[0x2000059f1:0x1:0x0](ffff88c5c9bb42c8), xattr lov 00000002:00010000:0.0:1767316589.782987:0:46199:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x2000059f1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 040000002 00010000:00010000:0.0:1767316589.783037:0:46199:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.783045:0:46199:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00000020:00100000:0.0:1767316589.783054:0:46199:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5ca304000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1767316589.783061:0:46199:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6402d80 x1853163991573248/t0(0) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 1512/131968 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1767316589.783069:0:46199:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00000100:00100000:0.0:1767316589.783091:0:46199:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991573248:192.168.204.145@tcp:101:lfs.0 00000100:00100000:0.0:1767316589.785169:0:46199:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2a80 going to sleep for 11 seconds 00000100:00100000:3.0:1767316589.789539:0:2480:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.145@tcp (source: 12345-192.168.204.145@tcp) 00000100:00100000:2.0:1767316589.789836:0:13885:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1853164044876160 00000100:00100000:2.0:1767316589.789847:0:13885:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1853164044876160 00000100:00100000:2.0:1767316589.789911:0:13885:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.145@tcp, seq: 1414 00000100:00100000:2.0:1767316589.789922:0:13885:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff88c5c572a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:4d658a6d-2c23-4e05-a7c2-999433a912b6+5:51433:x1853164044876160:12345-192.168.204.145@tcp:104: 00010000:00010000:2.0:1767316589.789939:0:13885:0:(ldlm_lockd.c:2505:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 2/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316589.790019:0:13885:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1767316589.790027:0:13885:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff88c5c572a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_002:4d658a6d-2c23-4e05-a7c2-999433a912b6+5:51433:x1853164044876160:12345-192.168.204.145@tcp:104: Request processed in 107us (499us total) trans 0 rc 0/0 00010000:00010000:0.0:1767316589.790042:0:15085:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1767316589.790049:0:15085:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 2/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.790075:0:15085:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c341d200 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:0.0:1767316589.790080:0:15085:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 2/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.790088:0:15085:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 2/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.790133:0:15085:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 3/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x428400000000 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316589.790158:0:15085:0:(vvp_object.c:114:vvp_conf_set()) [0x2000059f1:0x1:0x0]: losing layout lock 00010000:00010000:0.0:1767316589.790205:0:15085:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 2/0,0 mode: --/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.790212:0:15085:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1767316589.790233:0:15085:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 1/0,0 mode: --/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.790239:0:15085:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341d200/0x5295060e33d08a61 lrc: 0/0,0 mode: --/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x1b rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0x60377922a67795e9 expref: -99 pid: 46199 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316589.790259:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316589.790261:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316589.790263:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316589.790270:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767316589.790285:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2486:1853163991573376:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1767316589.790401:0:13885:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.145@tcp, seq: 1414 00000100:00100000:0.0:1767316589.795010:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2486:1853163991573376:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1767316589.799813:0:46199:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991573248:192.168.204.145@tcp:101:lfs.0 00010000:00010000:0.0:1767316589.799838:0:46199:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.799851:0:46199:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.799858:0:46199:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.799865:0:46199:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.799873:0:46199:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.799879:0:46199:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c341cc00 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:0.0:1767316589.799894:0:46199:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.799902:0:46199:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x206400000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.799909:0:46199:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x20e400000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.799915:0:46199:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1767316589.799921:0:46199:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 46199 initiator: MDT0 00010000:00010000:0.0:1767316589.799927:0:46199:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c341cc00/0x5295060e33d08a68 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 46199 initiator: MDT0 00000002:00100000:0.0:1767316589.799971:0:46199:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=800002b, status=0 req@ffff88c5c6402d80 x1853163991573248/t51539607556(51539607556) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 1512/1912 e 0 to 0 dl 1767316605 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1767316589.799983:0:46199:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6402d80 x1853163991573248/t51539607556(51539607556) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 1512/1912 e 0 to 0 dl 1767316605 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1767316589.800001:0:46199:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316589.800079:0:46199:0:(vvp_io.c:1854:vvp_io_init()) [0x2000059f1:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1767316589.800087:0:46199:0:(vvp_io.c:289:vvp_io_fini()) [0x2000059f1:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316589.800174:0:46199:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x2000059f1:0x1:0x0], updating i_size 0 00000002:00100000:0.0:1767316589.800190:0:46199:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6402d80 x1853163991573248/t51539607556(51539607556) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 1824/1912 e 0 to 0 dl 1767316605 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1767316589.800222:0:46199:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c6402d80 x1853163991573248/t51539607556(51539607556) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 1824/1912 e 0 to 0 dl 1767316605 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1767316589.800229:0:46199:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5ca304000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1767316589.800232:0:46199:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6401c00 x1853163991573632/t0(0) o35->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:23/10 lens 392/736 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1767316589.800247:0:46199:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991573632:192.168.204.145@tcp:35:lfs.0 00000100:00100000:0.0:1767316589.800308:0:46199:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2600 going to sleep for 11 seconds 00000100:00100000:0.0:1767316589.807845:0:46199:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991573632:192.168.204.145@tcp:35:lfs.0 00000002:00100000:0.0:1767316589.807879:0:46199:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c6402d80 x1853163991573248/t51539607556(51539607556) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 1824/1912 e 0 to 0 dl 1767316605 ref 2 fl Complete:RQU/604/0 rc 301/301 job:'lfs.0' uid:0 gid:0 projid:0 00000080:00200008:0.0:1767316589.807944:0:46199:0:(file.c:424:ll_file_release()) START file f203.replay-single:[0x2000059f1:0x1:0x0](ffff88c5c9bb42c8), flags 100001 00000002:00100000:0.0:1767316589.807998:0:46199:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c6400000 x1853163991572864/t51539607555(51539607555) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 536/608 e 0 to 0 dl 1767316605 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1767316589.808023:0:46199:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5ca304000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1767316589.808028:0:46199:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6403100 x1853163991573888/t0(0) o35->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:23/10 lens 392/736 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1767316589.808050:0:46199:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991573888:192.168.204.145@tcp:35:lfs.0 00000100:00100000:0.0:1767316589.808142:0:46199:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2600 going to sleep for 11 seconds 00000100:00100000:0.0:1767316589.816244:0:46199:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job lfs:4d658a6d-2c23-4e05-a7c2-999433a912b6:46199:1853163991573888:192.168.204.145@tcp:35:lfs.0 00000002:00100000:2.0:1767316589.816409:0:46199:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c6400000 x1853163991572864/t51539607555(51539607555) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 536/608 e 0 to 0 dl 1767316605 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00010000:00010000:1.0:1767316590.343278:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767316591.273474:0:2480:0:(events.c:108:reply_in_callback()) truncate req ffff88c5c65cf100 rpc 1376 - 1888+224 00000100:00100000:1.0:1767316591.275467:0:46197:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c7d54600 going to sleep for 1 seconds 00000100:00080000:1.0:1767316591.275497:0:46197:0:(client.c:3710:ptlrpc_set_mbits()) resend with new mbits old x1853163991572224 new x1853163991574016 00000100:00100000:1.0:1767316591.275604:0:46197:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c7d54600 going to sleep for 11 seconds 00000100:00100000:1.0:1767316591.279868:0:46197:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job stat:5398f0ac-a698-45f7-862d-28236c50e3c9:46197:1853163991572224:192.168.204.145@tcp:101:stat.0 00010000:00010000:1.0:1767316591.279895:0:46197:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316591.279928:0:46197:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 4/1,0 mode: --/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316591.279936:0:46197:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 4/1,0 mode: --/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316591.279947:0:46197:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 4/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316591.279957:0:46197:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 4/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316591.279969:0:46197:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 5/2,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316591.279976:0:46197:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 5/2,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316591.280001:0:46197:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 4/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316591.280013:0:46197:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cf100 x1853163991572224/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/1888 e 0 to 0 dl 1767316607 ref 1 fl Complete:RQU/602/0 rc 0/0 job:'stat.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316591.280023:0:46197:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 1304 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 3/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316591.280036:0:46197:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 3/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316591.280045:0:46197:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589957617/1 (0 0) 00000080:00200000:1.0:1767316591.280051:0:46197:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316591.280090:0:46197:0:(llite_lib.c:3217:ll_read_inode2()) VFS Op:inode=[0x2000059f1:0x1:0x0](ffff88c5c85d6c08) 00000080:00200000:1.0:1767316591.280254:0:46197:0:(file.c:6809:ll_layout_conf()) [0x2000059f1:0x1:0x0]: layout version change: 4294967294 -> 12 00000080:00010000:1.0:1767316591.280260:0:46197:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x2000059f1:0x1:0x0](ffff88c5c85d6c08) for lock 0x5295060e33d08a4c 00000080:00010000:1.0:1767316591.280271:0:46197:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000059f1:0x1:0x0](ffff88c5c85d6c08) 00000080:00010000:1.0:1767316591.280274:0:46197:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a4c from it ffffb10b01e57b20 00010000:00010000:1.0:1767316591.280278:0:46197:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 3/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316591.280286:0:46197:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 2/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00000080:00200008:1.0:1767316591.280328:0:46197:0:(file.c:6252:ll_getattr_dentry()) START file f203.replay-single:[0x2000059f1:0x1:0x0](ffff88c5c85d6c08), request_mask 4095, flags 0, foreign 0 00000080:00200000:1.0:1767316591.280335:0:46197:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x2000059f1:0x1:0x0](ffff88c5c85d6c08),name=f203.replay-single 00000002:00010000:1.0:1767316591.280354:0:46197:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x2000059f1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316591.280369:0:46197:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 2/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316591.280378:0:46197:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 2/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316591.280384:0:46197:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316591.280389:0:46197:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000059f1:0x1:0x0](ffff88c5c85d6c08) 00000080:00010000:1.0:1767316591.280392:0:46197:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x2000059f1:0x1:0x0](ffff88c5c85d6c08) for lock 0x5295060e33d08a4c 00000080:00010000:1.0:1767316591.280395:0:46197:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a4c from it ffffb10b01e57cb8 00010000:00010000:1.0:1767316591.280398:0:46197:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 3/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316591.280403:0:46197:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 2/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316591.280421:0:46197:0:(vvp_io.c:1854:vvp_io_init()) [0x2000059f1:0x1:0x0] ignore/verify layout 0/1, layout version 2 restore needed 0 00000001:00010000:1.0:1767316591.280430:0:46197:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x2000059f1:0x1:0x0] 00000008:00010000:1.0:1767316591.280446:0:46197:0:(lustre_osc.h:163:osc_enq2ldlm_flags()) flags: a 00000008:00010000:1.0:1767316591.280449:0:46197:0:(osc_lock.c:1248:osc_lock_init()) ### lock ffff88c5ca39ecb0, osc lock ffff88c5d8282be0, flags 0x400001002 00010000:00010000:1.0:1767316591.280493:0:46197:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767316591.280501:0:46197:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00000100:00100000:0.0:1767316591.280595:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991574144:192.168.204.145@tcp:101:stat.0 00000020:00200000:1.0:1767316591.281450:0:46197:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1767316591.297870:0:2486:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 6/2,0 mode: --/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316591.297900:0:2486:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 6/2,0 mode: --/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316591.297916:0:2486:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 6/2,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316591.297928:0:2486:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 6/2,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00000008:00010000:0.0:1767316591.297944:0:2486:0:(osc_lock.c:179:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 5/2,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00000020:00200000:0.0:1767316591.297955:0:2486:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:0.0:1767316591.297972:0:2486:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 6/2,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316591.298000:0:2486:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 5/1,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316591.298007:0:2486:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 5/1,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316591.298015:0:2486:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00000100:00100000:0.0:1767316591.298036:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991574144:192.168.204.145@tcp:101:stat.0 00000080:00200000:1.0:1767316591.298161:0:46197:0:(file.c:1686:ll_merge_attr_nolock()) [0x2000059f1:0x1:0x0] updating i_size 0 i_blocks 1 00000020:00010000:1.0:1767316591.298166:0:46197:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff88c5c572f478 (ffff88c5c60c16c8/0) at cl_lock_release():245 00000020:00010000:1.0:1767316591.298169:0:46197:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c572f478 (ffff88c5c60c16c8/0) at cl_lock_cancel():142 00000020:00010000:1.0:1767316591.298181:0:46197:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5ca39ecb0 (ffff88c5c60c12d8/1) at cl_lock_cancel():142 00000020:00010000:1.0:1767316591.298197:0:46197:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c572f478 (ffff88c5c60c16c8/0) at cl_lock_fini():72 00000020:00010000:1.0:1767316591.298199:0:46197:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5ca39ecb0 (ffff88c5c60c16c8/1) at cl_lock_fini():72 00000080:00200000:1.0:1767316591.298211:0:46197:0:(vvp_io.c:289:vvp_io_fini()) [0x2000059f1:0x1:0x0] ignore/verify layout 0/1, layout version 12 need write layout 0, restore needed 0, invalidate_lock 0 00010000:00010000:1.0:1767316591.368844:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316592.391918:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767316593.298029:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991574400:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.298327:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991574656:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.298356:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991574912:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.298393:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:3.0:1767316593.298400:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991574272:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.298429:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991574528:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.299240:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991574784:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.299299:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991575040:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.316591:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991574272:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.316619:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991574528:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.316632:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991574656:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.316644:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991574912:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316593.321329:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991575040:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316593.330090:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991574784:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316593.334177:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991574400:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316593.415400:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316594.439333:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316595.469076:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316596.491648:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316597.511877:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1767316597.935985:0:46636:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1767316597.935999:0:46636:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:3.0:1767316597.936021:0:46636:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1767316597.936087:0:46636:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46636 initiator: MDT0 00010000:00010000:3.0:1767316597.936128:0:46636:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46636 initiator: MDT0 00010000:00010000:3.0:1767316597.936139:0:46636:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46636 initiator: MDT0 00000100:00100000:3.0:1767316597.936166:0:46636:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job lsof:5398f0ac-a698-45f7-862d-28236c50e3c9:46636:1853163991575168:192.168.204.145@tcp:101:lsof.0 00000100:00100000:3.0:1767316597.936383:0:46636:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2600 going to sleep for 11 seconds 00000100:00100000:0.0:1767316597.940222:0:46636:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job lsof:5398f0ac-a698-45f7-862d-28236c50e3c9:46636:1853163991575168:192.168.204.145@tcp:101:lsof.0 00010000:00010000:0.0:1767316597.940249:0:46636:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316597.940262:0:46636:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316597.940272:0:46636:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316597.940283:0:46636:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316597.940295:0:46636:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316597.940317:0:46636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316597.940324:0:46636:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316597.940336:0:46636:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c572b100 x1853163991575168/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316613 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lsof.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316597.940350:0:46636:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316597.940376:0:46636:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1767316597.940384:0:46636:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316597.940419:0:46636:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 11776 00000080:00010000:0.0:1767316597.940427:0:46636:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a76 00000080:00010000:0.0:1767316597.940432:0:46636:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316597.940435:0:46636:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a76 from it ffffb10b01467cb8 00010000:00010000:0.0:1767316597.940439:0:46636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316597.940447:0:46636:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200008:3.0:1767316597.942785:0:46636:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c997b248), request_mask 2047, flags 0, foreign 0 00000080:00200000:3.0:1767316597.942800:0:46636:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248),name=/ 00000002:00010000:3.0:1767316597.942832:0:46636:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1767316597.942893:0:46636:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46636 initiator: MDT0 00010000:00010000:3.0:1767316597.942902:0:46636:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46636 initiator: MDT0 00010000:00010000:3.0:1767316597.942912:0:46636:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46636 initiator: MDT0 00000100:00100000:3.0:1767316597.942940:0:46636:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job lsof:4d658a6d-2c23-4e05-a7c2-999433a912b6:46636:1853163991575296:192.168.204.145@tcp:101:lsof.0 00000100:00100000:3.0:1767316597.943063:0:46636:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c56a2600 going to sleep for 11 seconds 00000100:00100000:3.0:1767316597.946898:0:46636:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5ca304000: committing for last_committed 51539607558 gen 1 00000100:00100000:3.0:1767316597.946969:0:46636:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job lsof:4d658a6d-2c23-4e05-a7c2-999433a912b6:46636:1853163991575296:192.168.204.145@tcp:101:lsof.0 00010000:00010000:3.0:1767316597.946989:0:46636:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316597.947001:0:46636:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316597.947012:0:46636:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316597.947021:0:46636:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316597.947032:0:46636:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316597.947040:0:46636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316597.947056:0:46636:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1767316597.947066:0:46636:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c5729880 x1853163991575296/t0(0) o101->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316613 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'lsof.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1767316597.947081:0:46636:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316597.947249:0:46636:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c9f99000 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:3.0:1767316597.947257:0:46636:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:3.0:1767316597.947281:0:46636:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 11776 00000080:00010000:3.0:1767316597.947286:0:46636:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) for lock 0x5295060e33d08a7d 00000080:00010000:3.0:1767316597.947292:0:46636:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) 00000080:00010000:3.0:1767316597.947295:0:46636:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a7d from it ffffb10b01467cb8 00010000:00010000:3.0:1767316597.947298:0:46636:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1767316597.947306:0:46636:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1767316598.409563:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991575424:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316598.410598:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991575680:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316598.410652:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991575936:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316598.414087:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2488:1853163991575424:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316598.414336:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1767316598.414351:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991575552:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316598.414561:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991575808:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316598.423420:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991575552:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316598.423440:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991575680:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316598.423451:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991575936:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316598.434989:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:4d658a6d-2c23-4e05-a7c2-999433a912b6:2488:1853163991575808:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316598.535364:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:0.0:1767316599.553036:0:46645:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c997b248), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1767316599.553050:0:46645:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248),name=/ 00000002:00010000:0.0:1767316599.553071:0:46645:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316599.553089:0:46645:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.553631:0:46645:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316599.553640:0:46645:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316599.553647:0:46645:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) 00000080:00010000:0.0:1767316599.553651:0:46645:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) for lock 0x5295060e33d08a7d 00000080:00010000:0.0:1767316599.553658:0:46645:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a7d from it ffffb10b01eefcb8 00010000:00010000:0.0:1767316599.553662:0:46645:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.553670:0:46645:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200008:0.0:1767316599.553856:0:46645:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c997b248), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1767316599.553862:0:46645:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248),name=/ 00000002:00010000:0.0:1767316599.553870:0:46645:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316599.553878:0:46645:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.553886:0:46645:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316599.553891:0:46645:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316599.553895:0:46645:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) 00000080:00010000:0.0:1767316599.553898:0:46645:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c997b248) for lock 0x5295060e33d08a7d 00000080:00010000:0.0:1767316599.553901:0:46645:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a7d from it ffffb10b01eefcb8 00010000:00010000:0.0:1767316599.553904:0:46645:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.553916:0:46645:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316599.564139:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:0.0:1767316599.583247:0:46645:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c997b248) 00000080:00200000:0.0:1767316599.583291:0:46645:0:(vvp_io.c:1854:vvp_io_init()) [0x2000059f1:0x1:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1767316599.583303:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00020000:00200000:0.0:1767316599.583314:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[1] flags 0x0 00020000:00200000:0.0:1767316599.583316:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[2] flags 0x0 00020000:00200000:0.0:1767316599.583317:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[3] flags 0x0 00020000:00200000:0.0:1767316599.583318:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[4] flags 0x0 00020000:00200000:0.0:1767316599.583319:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[5] flags 0x0 00020000:00200000:0.0:1767316599.583320:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[6] flags 0x0 00020000:00200000:0.0:1767316599.583322:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[7] flags 0x0 00020000:00200000:0.0:1767316599.583323:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[8] flags 0x0 00020000:00200000:0.0:1767316599.583324:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[9] flags 0x0 00020000:00200000:0.0:1767316599.583326:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[10] flags 0x0 00020000:00200000:0.0:1767316599.583327:0:46645:0:(lov_io.c:860:lov_io_iter_init()) component[11] flags 0x0 00000080:00200000:0.0:1767316599.583509:0:46645:0:(vvp_io.c:289:vvp_io_fini()) [0x2000059f1:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316599.583516:0:46645:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x2000059f1:0x1:0x0](ffff88c5c9bb42c8) 00000080:00200000:0.0:1767316599.583621:0:46645:0:(llite_lib.c:1568:ll_put_super()) VFS Op: cfg_instance lustre-client-ffff88c5ca304000 (sb ffff88c5ca304000) 00010000:00010000:0.0:1767316599.583658:0:46645:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583670:0:46645:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583676:0:46645:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000400000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583682:0:46645:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000400000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583733:0:46645:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1767316599.583740:0:46645:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583747:0:46645:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583752:0:46645:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583758:0:46645:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583770:0:46645:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1767316599.583781:0:15085:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1767316599.583786:0:15085:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000400000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583794:0:15085:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c606b200 already unused, calling callback (ffffffffc1436c20) 10000000:00010000:0.0:1767316599.583798:0:15085:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000400000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583806:0:15085:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1008400000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 10000000:00010000:0.0:1767316599.583812:0:15085:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1009400000000 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 10000000:01000000:0.0:1767316599.583818:0:15085:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:01000000:0.0:1767316599.583821:0:15085:0:(mgc_request.c:944:mgc_blocking_ast()) log lustre-client: stopping, won't requeue 00010000:00010000:0.0:1767316599.583862:0:15085:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5809400000020 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583868:0:15085:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1767316599.583891:0:15085:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5809400000020 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583897:0:15085:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.145@tcp lock: ffff88c5c606b200/0x5295060e33d089f1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5809400000020 nid: local remote: 0x60377922a6779580 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583908:0:15085:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1767316599.583911:0:15085:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583916:0:15085:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c341ca00 already unused, calling callback (ffffffffc1436c20) 10000000:00010000:0.0:1767316599.583919:0:15085:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.583925:0:15085:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1008400000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 10000000:00010000:0.0:1767316599.583931:0:15085:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1009400000000 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 10000000:01000000:0.0:1767316599.583936:0:15085:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 10000000:01000000:0.0:1767316599.583999:0:15085:0:(mgc_request.c:944:mgc_blocking_ast()) log lustre-cliir: stopping, won't requeue 10000000:01000000:0.0:1767316599.584002:0:15085:0:(mgc_request.c:130:config_log_put()) dropping config log lustre-cliir 00010000:00010000:0.0:1767316599.584016:0:15085:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.584022:0:15085:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1767316599.584028:0:15085:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.584044:0:15085:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.145@tcp lock: ffff88c5c341ca00/0x5295060e33d08a1b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0x60377922a6779587 expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.584055:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316599.584057:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316599.584059:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316599.584078:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991576192:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1767316599.584470:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767316599.584479:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991576064:192.168.204.145@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1767316599.584582:0:46645:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.584591:0:46645:0:(ldlm_lock.c:947:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.584596:0:46645:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.584603:0:46645:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316599.584618:0:46645:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 10000000:01000000:0.0:1767316599.584622:0:46645:0:(mgc_request.c:130:config_log_put()) dropping config log lustre-client 10000000:01000000:0.0:1767316599.584625:0:46645:0:(mgc_request.c:547:config_log_end()) end config log lustre-client (0) 00020000:00020000:0.0:1767316599.584912:0:46645:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff88c5ca304000: lov tgt 0 not cleaned! deathrow=0, lovrc=1 00010000:00010000:1.0:1767316599.585807:0:7704:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1767316599.585816:0:7704:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316599.585823:0:7704:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c341ce00 already unused, calling callback (ffffffffc1436c20) 10000000:00010000:1.0:1767316599.585826:0:7704:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316599.585834:0:7704:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1008400000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 10000000:00010000:1.0:1767316599.585839:0:7704:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1009400000000 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 10000000:01000000:1.0:1767316599.585844:0:7704:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 10000000:01000000:1.0:1767316599.585846:0:7704:0:(mgc_request.c:944:mgc_blocking_ast()) log params: stopping, won't requeue 10000000:01000000:1.0:1767316599.585849:0:7704:0:(mgc_request.c:130:config_log_put()) dropping config log params 00010000:00010000:1.0:1767316599.585877:0:7704:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316599.585882:0:7704:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316599.585918:0:7704:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316599.585924:0:7704:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.145@tcp lock: ffff88c5c341ce00/0x5295060e33d08a22 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0x60377922a677958e expref: -99 pid: 46159 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316599.585937:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316599.587509:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316599.587513:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316599.587540:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991576320:192.168.204.145@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1767316599.587637:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316599.591610:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991576192:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1767316599.593476:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991576064:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1767316599.598242:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991576320:192.168.204.145@tcp:103:ldlm_bl.0 00020000:01000000:1.0:1767316599.598278:0:46645:0:(lov_obd.c:589:lov_del_target()) uuid: lustre-OST0000_UUID idx: 0 gen: 1 exp: ffff88c5c2c4d800 active: 1 00020000:00020000:1.0:1767316599.598283:0:46645:0:(lov_obd.c:783:lov_cleanup()) lustre-clilov-ffff88c5ca304000: lov tgt 1 not cleaned! deathrow=1, lovrc=1 00020000:01000000:1.0:1767316599.598285:0:46645:0:(lov_obd.c:589:lov_del_target()) uuid: lustre-OST0001_UUID idx: 1 gen: 1 exp: ffff88c5c2c4a000 active: 1 00020000:01000000:1.0:1767316599.598288:0:46645:0:(lov_obd.c:64:lov_tgts_putref()) destroying 2 lov targets 00020000:01000000:1.0:1767316599.598292:0:46645:0:(lov_obd.c:611:__lov_del_obd()) Removing tgt lustre-OST0001_UUID : lustre-OST0001-osc-ffff88c5ca304000 00020000:01000000:1.0:1767316599.598294:0:46645:0:(lov_obd.c:248:lov_disconnect_obd()) lustre-clilov-ffff88c5ca304000: disconnecting target lustre-OST0001-osc-ffff88c5ca304000 00000100:00080000:1.0:1767316599.598301:0:46645:0:(pinger.c:396:ptlrpc_pinger_del_import()) removing pingable import 4d658a6d-2c23-4e05-a7c2-999433a912b6->lustre-OST0001_UUID 00000100:00080000:1.0:1767316599.598415:0:46645:0:(import.c:65:import_set_state_nolock()) ffff88c5c6a9c000 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00100000:0.0:1767316599.598695:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991576448:192.168.204.145@tcp:9:umount.0 00000100:00080000:0.0:1767316599.603431:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6a9c000 lustre-OST0001_UUID: changing import state from CONNECTING to CLOSED 00000100:00100000:0.0:1767316599.603512:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991576448:192.168.204.145@tcp:9:umount.0 00000100:00080000:1.0:1767316599.606265:0:46645:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0001_UUID INVALID 00000100:00100000:1.0:1767316599.606276:0:46645:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-ffff88c5ca304000: committing for last_committed 0 gen 2 00000100:00100000:1.0:1767316599.606286:0:46645:0:(import.c:321:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:1.0:1767316599.606425:0:46645:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x5295060e33d08a45 00000020:00000080:1.0:1767316599.606446:0:46645:0:(genops.c:904:class_export_put()) final put ffff88c5c2c4a000/LOV_OSC_UUID 00000020:01000000:1.0:1767316599.606583:0:46645:0:(obd_config.c:2361:class_manual_cleanup()) Manual cleanup of lustre-OST0001-osc-ffff88c5ca304000 (flags='') 00000020:00000080:1.0:1767316599.606590:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf004 00000020:00000080:1.0:1767316599.606601:0:46645:0:(obd_config.c:886:class_cleanup()) lustre-OST0001-osc-ffff88c5ca304000: forcing exports to disconnect: 0/3 00000020:00080000:1.0:1767316599.606605:0:46645:0:(genops.c:1575:class_disconnect_exports()) OBD device 10 (ffff88c5c5558000) has no exports 00000020:00000080:3.0:1767316599.609066:0:8:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c2c4a000/LOV_OSC_UUID for lustre-OST0001-osc-ffff88c5ca304000 00000008:01000000:1.0:1767316599.613315:0:46645:0:(obd_class.h:663:obd_cleanup_client_import()) lustre-OST0001-osc-ffff88c5ca304000: client import never connected 00000100:00080000:1.0:1767316599.613324:0:46645:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0001_UUID INVALID 00000100:00100000:1.0:1767316599.613329:0:46645:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-ffff88c5ca304000: committing for last_committed 0 gen 3 00000100:00020000:1.0:1767316599.613335:0:46645:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup 00000100:00100000:1.0:1767316599.620339:0:46645:0:(import.c:321:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000100:00020000:1.0:1767316599.620349:0:46645:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup 00000020:00000080:3.0:1767316599.620922:0:8:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff88c5c6a9c000 for lustre-OST0001-osc-ffff88c5ca304000 00000020:00000080:2.0:1767316599.621453:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf002 00000020:00000080:2.0:1767316599.621466:0:46645:0:(obd_config.c:814:class_detach()) detach on obd lustre-OST0001-osc-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:00000080:2.0:1767316599.621470:0:46645:0:(genops.c:904:class_export_put()) final put ffff88c5c6a9b000/4d658a6d-2c23-4e05-a7c2-999433a912b6 00000020:00000080:2.0:1767316599.621475:0:46645:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c6a9b000/4d658a6d-2c23-4e05-a7c2-999433a912b6 for lustre-OST0001-osc-ffff88c5ca304000 00000020:01000000:2.0:1767316599.621480:0:46645:0:(genops.c:443:class_free_dev()) finishing cleanup of obd lustre-OST0001-osc-ffff88c5ca304000 (4d658a6d-2c23-4e05-a7c2-999433a912b6) 00020000:01000000:2.0:1767316599.621661:0:46645:0:(lov_obd.c:611:__lov_del_obd()) Removing tgt lustre-OST0000_UUID : lustre-OST0000-osc-ffff88c5ca304000 00020000:01000000:2.0:1767316599.621664:0:46645:0:(lov_obd.c:248:lov_disconnect_obd()) lustre-clilov-ffff88c5ca304000: disconnecting target lustre-OST0000-osc-ffff88c5ca304000 00000100:00080000:2.0:1767316599.621670:0:46645:0:(pinger.c:396:ptlrpc_pinger_del_import()) removing pingable import 4d658a6d-2c23-4e05-a7c2-999433a912b6->lustre-OST0000_UUID 00000100:00080000:2.0:1767316599.621780:0:46645:0:(import.c:65:import_set_state_nolock()) ffff88c5c6a9a000 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00100000:0.0:1767316599.621847:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991576576:192.168.204.145@tcp:9:umount.0 00000100:00080000:0.0:1767316599.631510:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6a9a000 lustre-OST0000_UUID: changing import state from CONNECTING to CLOSED 00000100:00100000:0.0:1767316599.631538:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991576576:192.168.204.145@tcp:9:umount.0 00000100:00080000:2.0:1767316599.631610:0:46645:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0000_UUID INVALID 00000100:00100000:2.0:1767316599.631615:0:46645:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-ffff88c5ca304000: committing for last_committed 0 gen 2 00000100:00100000:2.0:1767316599.631620:0:46645:0:(import.c:321:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:2.0:1767316599.631769:0:46645:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x5295060e33d08a3e 00000020:00000080:2.0:1767316599.631787:0:46645:0:(genops.c:904:class_export_put()) final put ffff88c5c2c4d800/LOV_OSC_UUID 00000020:01000000:2.0:1767316599.631845:0:46645:0:(obd_config.c:2361:class_manual_cleanup()) Manual cleanup of lustre-OST0000-osc-ffff88c5ca304000 (flags='') 00000020:00000080:2.0:1767316599.631852:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf004 00000020:00000080:2.0:1767316599.631863:0:46645:0:(obd_config.c:886:class_cleanup()) lustre-OST0000-osc-ffff88c5ca304000: forcing exports to disconnect: 0/3 00000020:00080000:2.0:1767316599.631868:0:46645:0:(genops.c:1575:class_disconnect_exports()) OBD device 9 (ffff88c5c5008000) has no exports 00000020:00000080:3.0:1767316599.631950:0:8:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c2c4d800/LOV_OSC_UUID for lustre-OST0000-osc-ffff88c5ca304000 00000008:01000000:2.0:1767316599.632034:0:46645:0:(obd_class.h:663:obd_cleanup_client_import()) lustre-OST0000-osc-ffff88c5ca304000: client import never connected 00000100:00080000:2.0:1767316599.632038:0:46645:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0000_UUID INVALID 00000100:00100000:2.0:1767316599.632040:0:46645:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-ffff88c5ca304000: committing for last_committed 0 gen 3 00000100:00020000:2.0:1767316599.632044:0:46645:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup 00000100:00100000:2.0:1767316599.632045:0:46645:0:(import.c:321:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000100:00020000:2.0:1767316599.632047:0:46645:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup 00000020:00000080:2.0:1767316599.632401:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf002 00000020:00000080:2.0:1767316599.632407:0:46645:0:(obd_config.c:814:class_detach()) detach on obd lustre-OST0000-osc-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:00000080:2.0:1767316599.632469:0:46645:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x5295060e33d08a37 00000020:00000080:2.0:1767316599.632483:0:46645:0:(genops.c:904:class_export_put()) final put ffff88c5c2c4b000/4d658a6d-2c23-4e05-a7c2-999433a912b6 00000100:00080000:2.0:1767316599.632762:0:46645:0:(pinger.c:396:ptlrpc_pinger_del_import()) removing pingable import 4d658a6d-2c23-4e05-a7c2-999433a912b6->lustre-MDT0000_UUID 00010000:00010000:2.0:1767316599.632880:0:46645:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316599.632927:0:46645:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316599.632935:0:46645:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1767316599.632963:0:46645:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c5bd80 going to sleep for 0 seconds 00000100:00100000:2.0:1767316599.632970:0:46645:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job umount:4d658a6d-2c23-4e05-a7c2-999433a912b6:46645:1853163991576704:192.168.204.145@tcp:103:umount.0 00010000:00010000:2.0:1767316599.632978:0:46645:0:(ldlm_request.c:1494:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway 00010000:00010000:2.0:1767316599.632985:0:46645:0:(ldlm_request.c:2263:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 00010000:00010000:2.0:1767316599.632988:0:46645:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5ca304000 lock: ffff88c5c4d1ae00/0x5295060e33d08a7d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677963d expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000100:00080000:2.0:1767316599.634053:0:46645:0:(import.c:65:import_set_state_nolock()) ffff88c5c6a9e800 lustre-MDT0000_UUID: changing import state from FULL to CONNECTING 00000100:00100000:1.0:1767316599.634263:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991576832:192.168.204.145@tcp:39:umount.0 00000020:00000080:3.0:1767316599.634377:0:8:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff88c5c6a9a000 for lustre-OST0000-osc-ffff88c5ca304000 00000020:00000080:3.0:1767316599.634387:0:8:0:(genops.c:904:class_export_put()) final put ffff88c5c6a9c800/4d658a6d-2c23-4e05-a7c2-999433a912b6 00000020:00000080:3.0:1767316599.634391:0:8:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c6a9c800/4d658a6d-2c23-4e05-a7c2-999433a912b6 for lustre-OST0000-osc-ffff88c5ca304000 00000020:01000000:3.0:1767316599.634399:0:8:0:(genops.c:443:class_free_dev()) finishing cleanup of obd lustre-OST0000-osc-ffff88c5ca304000 (4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:00000080:3.0:1767316599.634637:0:8:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c2c4b000/4d658a6d-2c23-4e05-a7c2-999433a912b6 for lustre-clilov-ffff88c5ca304000 00000100:00080000:1.0:1767316599.651596:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6a9e800 lustre-MDT0000_UUID: changing import state from CONNECTING to CLOSED 00000100:00100000:1.0:1767316599.651628:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:4d658a6d-2c23-4e05-a7c2-999433a912b6:2485:1853163991576832:192.168.204.145@tcp:39:umount.0 00000100:00080000:2.0:1767316599.651690:0:46645:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:2.0:1767316599.651695:0:46645:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5ca304000: committing for last_committed 51539607558 gen 2 00000100:00100000:2.0:1767316599.651712:0:46645:0:(client.c:3119:ptlrpc_free_committed()) @@@ free request with old gen req@ffff88c5c57d7800 x1853163991576832/t51539607559(51539607559) o39->lustre-MDT0000-mdc-ffff88c5ca304000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 0 dl 1767316609 ref 1 fl Complete:RNQU/204/0 rc 0/0 job:'umount.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316599.651734:0:46645:0:(import.c:321:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00000080:2.0:1767316599.653647:0:46645:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x5295060e33d08a30 00000020:00000080:2.0:1767316599.653663:0:46645:0:(genops.c:904:class_export_put()) final put ffff88c5c2c4f000/4d658a6d-2c23-4e05-a7c2-999433a912b6 00000020:00000080:2.0:1767316599.653719:0:46645:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x5295060e33d08a29 00000020:00000080:2.0:1767316599.653724:0:46645:0:(genops.c:904:class_export_put()) final put ffff88c5c2c4e800/4d658a6d-2c23-4e05-a7c2-999433a912b6 00000020:01000000:2.0:1767316599.653733:0:46645:0:(obd_config.c:2361:class_manual_cleanup()) Manual cleanup of lustre-clilov-ffff88c5ca304000 (flags='') 00000020:00000080:2.0:1767316599.653742:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf004 00000020:00000080:2.0:1767316599.653749:0:46645:0:(obd_config.c:886:class_cleanup()) lustre-clilov-ffff88c5ca304000: forcing exports to disconnect: 0/1 00000020:00080000:2.0:1767316599.653768:0:46645:0:(genops.c:1575:class_disconnect_exports()) OBD device 6 (ffff88c5c500b1b0) has no exports 00000020:00000080:2.0:1767316599.653795:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf002 00000020:00000080:2.0:1767316599.653801:0:46645:0:(obd_config.c:814:class_detach()) detach on obd lustre-clilov-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:00000080:2.0:1767316599.653804:0:46645:0:(genops.c:904:class_export_put()) final put ffff88c5c6a9f800/4d658a6d-2c23-4e05-a7c2-999433a912b6 00000020:00000080:2.0:1767316599.653806:0:46645:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c6a9f800/4d658a6d-2c23-4e05-a7c2-999433a912b6 for lustre-clilov-ffff88c5ca304000 00000020:01000000:2.0:1767316599.653821:0:46645:0:(genops.c:443:class_free_dev()) finishing cleanup of obd lustre-clilov-ffff88c5ca304000 (4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:01000000:2.0:1767316599.653831:0:46645:0:(obd_config.c:2361:class_manual_cleanup()) Manual cleanup of lustre-clilmv-ffff88c5ca304000 (flags='') 00000020:00000080:2.0:1767316599.653834:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf004 00000020:00000080:2.0:1767316599.653837:0:46645:0:(obd_config.c:886:class_cleanup()) lustre-clilmv-ffff88c5ca304000: forcing exports to disconnect: 0/2 00000020:00080000:2.0:1767316599.653840:0:46645:0:(genops.c:1575:class_disconnect_exports()) OBD device 7 (ffff88c5c50098d8) has no exports 00000020:00000080:2.0:1767316599.653895:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf002 00000020:00000080:2.0:1767316599.653898:0:46645:0:(obd_config.c:814:class_detach()) detach on obd lustre-clilmv-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:01000000:2.0:1767316599.653901:0:46645:0:(obd_config.c:2361:class_manual_cleanup()) Manual cleanup of lustre-MDT0000-mdc-ffff88c5ca304000 (flags='') 00000020:00000080:2.0:1767316599.653904:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf004 00000020:00000080:2.0:1767316599.653906:0:46645:0:(obd_config.c:886:class_cleanup()) lustre-MDT0000-mdc-ffff88c5ca304000: forcing exports to disconnect: 0/3 00000020:00080000:2.0:1767316599.653908:0:46645:0:(genops.c:1575:class_disconnect_exports()) OBD device 8 (ffff88c5c500ca88) has no exports 00000020:00000080:3.0:1767316599.654158:0:8649:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c2c4f000/4d658a6d-2c23-4e05-a7c2-999433a912b6 for lustre-MDT0000-mdc-ffff88c5ca304000 00000020:00000080:3.0:1767316599.654165:0:8649:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c2c4e800/4d658a6d-2c23-4e05-a7c2-999433a912b6 for lustre-clilmv-ffff88c5ca304000 00000020:00000080:3.0:1767316599.654169:0:8649:0:(genops.c:904:class_export_put()) final put ffff88c5c6a99800/4d658a6d-2c23-4e05-a7c2-999433a912b6 00000020:00000080:3.0:1767316599.654171:0:8649:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c6a99800/4d658a6d-2c23-4e05-a7c2-999433a912b6 for lustre-clilmv-ffff88c5ca304000 00000020:01000000:3.0:1767316599.654175:0:8649:0:(genops.c:443:class_free_dev()) finishing cleanup of obd lustre-clilmv-ffff88c5ca304000 (4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000008:01000000:2.0:1767316599.655022:0:46645:0:(obd_class.h:663:obd_cleanup_client_import()) lustre-MDT0000-mdc-ffff88c5ca304000: client import never connected 00000100:00080000:2.0:1767316599.655029:0:46645:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:2.0:1767316599.655034:0:46645:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5ca304000: committing for last_committed 51539607558 gen 3 00000100:00020000:2.0:1767316599.655039:0:46645:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup 00000100:00100000:2.0:1767316599.655042:0:46645:0:(import.c:321:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000100:00020000:2.0:1767316599.655044:0:46645:0:(obd_class.h:479:obd_check_dev()) Device 8 not setup 00000020:00000080:2.0:1767316599.655402:0:46645:0:(obd_config.c:1389:class_process_config()) processing cmd: cf002 00000020:00000080:2.0:1767316599.655409:0:46645:0:(obd_config.c:814:class_detach()) detach on obd lustre-MDT0000-mdc-ffff88c5ca304000 (uuid 4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000080:02000400:2.0:1767316599.655414:0:46645:0:(llite_lib.c:1613:ll_put_super()) Unmounted lustre-client 00000020:01000000:2.0:1767316599.658901:0:46645:0:(obd_config.c:1123:class_del_profile()) Del profile lustre-client 00000020:00000080:1.0:1767316599.660798:0:8:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff88c5c6a9e800 for lustre-MDT0000-mdc-ffff88c5ca304000 00000020:00000080:1.0:1767316599.660815:0:8:0:(genops.c:904:class_export_put()) final put ffff88c5c6a98800/4d658a6d-2c23-4e05-a7c2-999433a912b6 00000020:00000080:1.0:1767316599.660819:0:8:0:(genops.c:863:class_export_destroy()) destroying export ffff88c5c6a98800/4d658a6d-2c23-4e05-a7c2-999433a912b6 for lustre-MDT0000-mdc-ffff88c5ca304000 00000020:01000000:1.0:1767316599.660831:0:8:0:(genops.c:443:class_free_dev()) finishing cleanup of obd lustre-MDT0000-mdc-ffff88c5ca304000 (4d658a6d-2c23-4e05-a7c2-999433a912b6) 00000020:01000004:2.0:1767316599.661833:0:46645:0:(obd_mount.c:875:lustre_common_put_super()) dropping sb ffff88c5ca304000 00000020:01000004:2.0:1767316599.661840:0:46645:0:(obd_mount.c:569:lustre_stop_mgc()) MGC still has 1 references. 00000020:01000004:2.0:1767316599.661842:0:46645:0:(obd_mount.c:888:lustre_common_put_super()) MGC still in use 00000020:01000004:2.0:1767316599.661844:0:46645:0:(obd_mount.c:705:lustre_put_lsi()) put ffff88c5ca304000 1 00000020:01000004:2.0:1767316599.661847:0:46645:0:(obd_mount.c:637:lustre_free_lsi()) Freeing lsi ffff88c5ca306800 00010000:00010000:2.0:1767316600.583601:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1767316600.972576:0:46776:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test complete, duration 1374 sec ======== 20:16:39 (1767316599) 00010000:00010000:2.0:1767316601.610496:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316602.632142:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1767316603.116883:0:46920:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: === replay-single: start cleanup 20:16:41 (1767316601) === 00000080:00200000:1.0:1767316603.293773:0:9400:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316603.293801:0:9400:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.293819:0:9400:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.293835:0:9400:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.293843:0:9400:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.293850:0:9400:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.293854:0:9400:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a76 00000080:00010000:1.0:1767316603.293872:0:9400:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a76 from it ffffb10b01c63bb0 00010000:00010000:1.0:1767316603.293877:0:9400:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.293885:0:9400:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.293903:0:9400:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 44 00000080:00200008:1.0:1767316603.293912:0:9400:0:(file.c:1001:ll_file_open()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 304000 00000080:00200008:1.0:1767316603.293928:0:9400:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 2047, flags 0, foreign 0 00000002:00010000:1.0:1767316603.293942:0:9400:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: readdir flags 0 00010000:00010000:1.0:1767316603.293949:0:9400:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.293957:0:9400:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316603.294614:0:9400:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job bash:5398f0ac-a698-45f7-862d-28236c50e3c9:9400:1853163991576960:192.168.204.145@tcp:37:bash.0 00000100:00100000:1.0:1767316603.294746:0:9400:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.320735:0:9400:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job bash:5398f0ac-a698-45f7-862d-28236c50e3c9:9400:1853163991576960:192.168.204.145@tcp:37:bash.0 00010000:00010000:1.0:1767316603.323930:0:9400:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.323951:0:9400:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.323985:0:9400:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316603.324005:0:9400:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.324029:0:9400:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.324041:0:9400:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.324048:0:9400:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.324053:0:9400:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.324057:0:9400:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a76 00000080:00010000:1.0:1767316603.324063:0:9400:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a76 from it ffffb10b01c63cd8 00010000:00010000:1.0:1767316603.324066:0:9400:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.324072:0:9400:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.324327:0:9400:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) pos/size0/11776 32bit_api 0 00000080:00200000:1.0:1767316603.324385:0:9400:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8) pos/size18446744073709551614/11776 32bit_api 0 00000080:00200008:1.0:1767316603.324402:0:9400:0:(file.c:424:ll_file_release()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags 304000 00000080:00200000:2.0:1767316603.328206:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316603.328231:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.328248:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.328264:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.328272:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316603.328279:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316603.328283:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a76 00000080:00010000:2.0:1767316603.328289:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a76 from it ffffb10b019a7b58 00010000:00010000:2.0:1767316603.328293:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.328301:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.328315:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316603.328326:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d103.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:2.0:1767316603.328329:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d103.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:2.0:1767316603.328341:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d103.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.328390:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.328398:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.328417:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316603.328444:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577088:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316603.332075:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.357376:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577088:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316603.357404:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357427:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357435:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357446:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357457:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357470:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357478:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357485:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316603.357511:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d7800 x1853163991577088/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316603.357524:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357535:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1034 (0 0) 00000080:00200000:2.0:1767316603.357542:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316603.357562:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x40a:0x0], updating i_size 11776 00000080:00200000:2.0:1767316603.357566:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x40a:0x0](ffff88c5c85d5348): rc = 0 00000080:00010000:2.0:1767316603.357571:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x40a:0x0](ffff88c5c85d5348) for lock 0x5295060e33d08a84 00000080:00010000:2.0:1767316603.357584:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x40a:0x0](ffff88c5c85d5348) 00000080:00010000:2.0:1767316603.357588:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a84 from it ffffb10b019a7b20 00010000:00010000:2.0:1767316603.357591:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357599:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200008:2.0:1767316603.357646:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d103.replay-single:[0x200000bd1:0x40a:0x0](ffff88c5c85d5348), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1767316603.357655:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x40a:0x0](ffff88c5c85d5348),name=d103.replay-single 00000002:00010000:2.0:1767316603.357678:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x40a:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.357686:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357697:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.357714:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316603.357719:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x40a:0x0](ffff88c5c85d5348) 00000080:00010000:2.0:1767316603.357722:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x40a:0x0](ffff88c5c85d5348) for lock 0x5295060e33d08a84 00000080:00010000:2.0:1767316603.357726:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a84 from it ffffb10b019a7cb8 00010000:00010000:2.0:1767316603.357730:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357736:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.357831:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316603.357839:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.357847:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357855:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.357861:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316603.357865:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316603.357867:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a76 00000080:00010000:2.0:1767316603.357871:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a76 from it ffffb10b019a7bf8 00010000:00010000:2.0:1767316603.357874:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.357881:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.357899:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316603.357906:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d103.replay-single, flags=266 00000080:00200000:2.0:1767316603.357912:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x40a:0x0](ffff88c5c85d5348), inode mode 41ed mask 44 00000080:00200008:2.0:1767316603.357920:0:46936:0:(file.c:1001:ll_file_open()) START file d103.replay-single:[0x200000bd1:0x40a:0x0](ffff88c5c85d5348), flags 704400 00000002:00010000:2.0:1767316603.357934:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x40a:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100240704401 00010000:00010000:2.0:1767316603.357982:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.357988:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:2.0:1767316603.357999:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1767316603.358006:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d7800 x1853163991577344/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1767316603.358015:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316603.358034:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577344:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316603.359331:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.371402:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577344:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316603.371427:0:46936:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.371441:0:46936:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.371449:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.371476:0:46936:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.371485:0:46936:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.371491:0:46936:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c606be00 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:2.0:1767316603.371496:0:46936:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.371503:0:46936:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.371509:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.371516:0:46936:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1767316603.371523:0:46936:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.371528:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606be00/0x5295060e33d08a8b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00000002:00100000:2.0:1767316603.371550:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c57d7800 x1853163991577344/t51539607560(51539607560) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1767316603.371563:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c57d7800 x1853163991577344/t51539607560(51539607560) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1767316603.371576:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1767316603.371596:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x40a:0x0], updating i_size 11776 00000002:00100000:2.0:1767316603.371613:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c57d7800 x1853163991577344/t51539607560(51539607560) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:2.0:1767316603.371695:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d103.replay-single:[0x200000bd1:0x40a:0x0](ffff88c5c85d5348), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1767316603.371702:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x40a:0x0](ffff88c5c85d5348),name=d103.replay-single 00000002:00010000:2.0:1767316603.371725:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x40a:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.371740:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.371753:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.371760:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316603.371766:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x40a:0x0](ffff88c5c85d5348) 00000080:00010000:2.0:1767316603.371770:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x40a:0x0](ffff88c5c85d5348) for lock 0x5295060e33d08a84 00000080:00010000:2.0:1767316603.371775:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a84 from it ffffb10b019a7cd8 00010000:00010000:2.0:1767316603.371778:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.371786:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.371859:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x40a:0x0](ffff88c5c85d5348) pos/size0/11776 32bit_api 0 00000002:00010000:2.0:1767316603.371873:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x40a:0x0]) in obj [0x200000bd1:0x40a:0x0], intent: readdir flags 0 00010000:00010000:2.0:1767316603.371880:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.371888:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1767316603.372795:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577472:192.168.204.145@tcp:37:rm.0 00000100:00100000:2.0:1767316603.373023:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.396668:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577472:192.168.204.145@tcp:37:rm.0 00010000:00010000:2.0:1767316603.397411:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.397428:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1767316603.397588:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x40a:0x0](ffff88c5c85d5348) pos/size18446744073709551614/11776 32bit_api 0 00000080:00200008:3.0:1767316603.397686:0:46936:0:(file.c:424:ll_file_release()) START file d103.replay-single:[0x200000bd1:0x40a:0x0](ffff88c5c85d5348), flags 704000 00000002:00100000:3.0:1767316603.397755:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c57d7800 x1853163991577344/t51539607560(51539607560) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1767316603.397766:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1767316603.397771:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c5729880 x1853163991577728/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1767316603.397796:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577728:192.168.204.145@tcp:35:rm.0 00000100:00100000:3.0:1767316603.397933:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426a80 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.400928:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577728:192.168.204.145@tcp:35:rm.0 00000002:00100000:1.0:1767316603.400964:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c57d7800 x1853163991577344/t51539607560(51539607560) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316603.401034:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316603.401052:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.401070:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.401083:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.401561:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.401568:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.401572:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a76 00000080:00010000:1.0:1767316603.401581:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a76 from it ffffb10b019a7c10 00010000:00010000:1.0:1767316603.401585:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.401593:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.401608:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.401619:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d103.replay-single, flags=0 00000080:00200000:1.0:1767316603.401625:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316603.401631:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.401646:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.401655:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.401660:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.401664:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.401666:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a76 00000080:00010000:1.0:1767316603.401669:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a76 from it ffffb10b019a7d88 00010000:00010000:1.0:1767316603.401672:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.401678:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.401694:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:1.0:1767316603.401700:0:46936:0:(namei.c:2264:ll_rmdir()) VFS Op:name=d103.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:1.0:1767316603.401723:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.401801:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.401862:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.401869:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.401874:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1767316603.401877:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a800/0x5295060e33d08a84 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x40a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779659 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.401891:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1bc00/0x5295060e33d08a76 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677962f expref: -99 pid: 46636 timeout: 0 lvb_type: 0 00000020:00100000:1.0:1767316603.401908:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316603.401916:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cdc00 x1853163991577984/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316603.401937:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577984:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316603.402029:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.413058:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991577984:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316603.413157:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x40a:0x0](ffff88c5c85d5348) 00000080:00200000:1.0:1767316603.413522:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316603.413541:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.413600:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.413611:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.413621:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.413654:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578112:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.413729:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.416171:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578112:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.416196:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.416209:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.416221:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.416234:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.416528:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.416563:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.416571:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.416584:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cd500 x1853163991578112/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.416598:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.416611:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1767316603.416618:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.416657:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 11776 00000080:00010000:1.0:1767316603.416664:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a92 00000080:00010000:1.0:1767316603.416669:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.416672:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a92 from it ffffb10b019a7b58 00010000:00010000:1.0:1767316603.416675:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.416683:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.416710:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.416739:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d130b.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:1.0:1767316603.416743:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d130b.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:1.0:1767316603.416763:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d130b.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.416810:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.416818:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.416827:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.416848:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578240:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.416932:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.430794:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578240:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.430820:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.430843:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.430852:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.430862:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.430873:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.430885:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.430893:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.430914:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.430926:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cd500 x1853163991578240/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.430939:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.430949:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1068 (0 0) 00000080:00200000:1.0:1767316603.430956:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316603.430976:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x42c:0x0], updating i_size 11776 00000080:00200000:1.0:1767316603.430980:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8): rc = 0 00000080:00010000:1.0:1767316603.430985:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) for lock 0x5295060e33d08a99 00000080:00010000:1.0:1767316603.430997:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) 00000080:00010000:1.0:1767316603.431001:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a99 from it ffffb10b019a7b20 00010000:00010000:1.0:1767316603.431004:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.431012:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316603.431048:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d130b.replay-single:[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.431055:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8),name=d130b.replay-single 00000002:00010000:1.0:1767316603.431075:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x42c:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.431083:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.431145:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.431153:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.431158:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) 00000080:00010000:1.0:1767316603.431161:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) for lock 0x5295060e33d08a99 00000080:00010000:1.0:1767316603.431165:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a99 from it ffffb10b019a7cb8 00010000:00010000:1.0:1767316603.431168:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.431174:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.431225:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316603.431232:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.431239:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.431252:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.431259:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.431263:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.431266:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a92 00000080:00010000:1.0:1767316603.431270:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a92 from it ffffb10b019a7bf8 00010000:00010000:1.0:1767316603.431273:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.431289:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.431299:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.431307:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d130b.replay-single, flags=266 00000080:00200000:1.0:1767316603.431312:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), inode mode 41ed mask 44 00000080:00200008:1.0:1767316603.431320:0:46936:0:(file.c:1001:ll_file_open()) START file d130b.replay-single:[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), flags 704400 00000002:00010000:1.0:1767316603.431334:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x42c:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100240704401 00010000:00010000:1.0:1767316603.431392:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.431400:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:1.0:1767316603.431410:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316603.431417:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cd500 x1853163991578496/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316603.431425:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.431444:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578496:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.431609:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.437929:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578496:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.437952:0:46936:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.437965:0:46936:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.437991:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.437997:0:46936:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.438004:0:46936:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.438010:0:46936:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5d191ba00 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:1.0:1767316603.438014:0:46936:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.438021:0:46936:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.438027:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.438033:0:46936:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1767316603.438039:0:46936:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.438045:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00000002:00100000:1.0:1767316603.438063:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c65cd500 x1853163991578496/t51539607563(51539607563) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316603.438076:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cd500 x1853163991578496/t51539607563(51539607563) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316603.438089:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.438151:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x42c:0x0], updating i_size 11776 00000002:00100000:1.0:1767316603.438168:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cd500 x1853163991578496/t51539607563(51539607563) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:1.0:1767316603.438231:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d130b.replay-single:[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.438237:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8),name=d130b.replay-single 00000002:00010000:1.0:1767316603.438255:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x42c:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.439451:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.439465:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.439474:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.439485:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) 00000080:00010000:1.0:1767316603.439489:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) for lock 0x5295060e33d08a99 00000080:00010000:1.0:1767316603.439498:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a99 from it ffffb10b019a7cd8 00010000:00010000:1.0:1767316603.439501:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.439510:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.439597:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) pos/size0/11776 32bit_api 0 00000002:00010000:1.0:1767316603.439613:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x42c:0x0]) in obj [0x200000bd1:0x42c:0x0], intent: readdir flags 0 00010000:00010000:1.0:1767316603.439620:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.439627:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316603.441388:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578624:192.168.204.145@tcp:37:rm.0 00000100:00100000:1.0:1767316603.441503:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.475664:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578624:192.168.204.145@tcp:37:rm.0 00010000:00010000:1.0:1767316603.476390:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.476407:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316603.476471:0:46936:0:(file.c:424:ll_file_release()) START file d130b.replay-single:[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), flags 704000 00000002:00100000:1.0:1767316603.476538:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cd500 x1853163991578496/t51539607563(51539607563) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1767316603.476551:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1767316603.476556:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cce00 x1853163991578880/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1767316603.476579:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578880:192.168.204.145@tcp:35:rm.0 00000100:00100000:1.0:1767316603.476681:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.485780:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991578880:192.168.204.145@tcp:35:rm.0 00000002:00100000:1.0:1767316603.485819:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cd500 x1853163991578496/t51539607563(51539607563) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316603.485903:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316603.485921:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.485943:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.485976:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.485985:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.485991:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.485994:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a92 00000080:00010000:1.0:1767316603.486001:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a92 from it ffffb10b019a7bf8 00010000:00010000:1.0:1767316603.486006:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.486015:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.486028:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.486035:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d130b.replay-single, flags=266 00000080:00200000:1.0:1767316603.486041:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), inode mode 41ed mask 44 00000080:00200008:1.0:1767316603.486062:0:46936:0:(file.c:1001:ll_file_open()) START file d130b.replay-single:[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), flags 704400 00000002:00010000:1.0:1767316603.486078:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x42c:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0104240704401 00010000:00010000:1.0:1767316603.486186:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.486194:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:1.0:1767316603.486206:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316603.486213:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cce00 x1853163991579136/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316603.486223:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.486251:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991579136:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.486431:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.498849:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991579136:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.498885:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.498897:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.498909:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.498919:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.498934:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=200002b, status=0 req@ffff88c5c65cce00 x1853163991579136/t51539607565(51539607565) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316603.498947:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cce00 x1853163991579136/t51539607565(51539607565) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.498958:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.498968:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 16 res 8589937617/1068 (0 0) 00000080:00200000:1.0:1767316603.498974:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.498996:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x42c:0x0], updating i_size 11776 00000080:00010000:1.0:1767316603.499001:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) for lock 0x5295060e33d08aa7 00000080:00010000:1.0:1767316603.499023:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08aa7 from it ffffb10b019a7bd0 00010000:00010000:1.0:1767316603.499027:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.499037:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.499061:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cce00 x1853163991579136/t51539607565(51539607565) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:1.0:1767316603.499165:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d130b.replay-single:[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.499172:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8),name=d130b.replay-single 00000002:00010000:1.0:1767316603.499193:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x42c:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.499203:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.499214:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.499220:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.499226:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) 00000080:00010000:1.0:1767316603.499229:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) for lock 0x5295060e33d08a99 00000080:00010000:1.0:1767316603.499233:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a99 from it ffffb10b019a7cd8 00010000:00010000:1.0:1767316603.499235:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.499242:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.499288:0:46936:0:(llite_lib.c:2766:ll_statfs()) VFS Op:sb=lustre (ffff88c5c68bd000) d130b.replay-single proj=0 00000080:00000004:1.0:1767316603.499292:0:46936:0:(obd_class.h:1083:obd_statfs()) lustre-clilmv-ffff88c5c68bd000: age 893, max_age 1801 00800000:00000004:1.0:1767316603.499300:0:46936:0:(obd_class.h:1083:obd_statfs()) lustre-MDT0000-mdc-ffff88c5c68bd000: age 1779, max_age 1801 00000100:00100000:1.0:1767316603.499336:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991579264:192.168.204.145@tcp:41:rm.0 00000100:00100000:1.0:1767316603.499409:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25ecc0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.508170:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991579264:192.168.204.145@tcp:41:rm.0 00800000:00000004:1.0:1767316603.508200:0:46936:0:(obd_class.h:1109:obd_statfs()) lustre-MDT0000-mdc-ffff88c5c68bd000: update ffff88c5d0093520 cache blocks 1882112/1885696 objects 235392/235760 00000080:00000004:1.0:1767316603.508207:0:46936:0:(obd_class.h:1109:obd_statfs()) lustre-clilmv-ffff88c5c68bd000: update ffff88c5d00966d0 cache blocks 1882112/1885696 objects 235392/235760 00000080:00000004:1.0:1767316603.508212:0:46936:0:(llite_lib.c:2629:ll_statfs_internal()) MDC blocks 1882112/1885696 objects 235392/235760 00000080:00200000:1.0:1767316603.508251:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) pos/size0/11776 32bit_api 0 00000080:00200000:1.0:1767316603.508466:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) pos/size18446744073709551614/11776 32bit_api 0 00000080:00200000:1.0:1767316603.508490:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.508516:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=f130b.replay-single, dir=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), flags=0 00000080:00200000:1.0:1767316603.508520:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f130b.replay-single, dir=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), intent=getattr 00000002:00010000:1.0:1767316603.508540:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f130b.replay-single,[0x0:0x0:0x0]) in obj [0x200000bd1:0x42c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.508592:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.508602:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.508610:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.508630:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991579392:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.508728:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.527001:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991579392:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.527029:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.527061:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.527069:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.527080:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.527090:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000100:00080000:2.0:1767316603.527230:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:02000000:2.0:1767316603.527234:0:8:0:(import.c:2080:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff88c5c68bd000: disconnect after 23s idle 00000100:00100000:3.0:1767316603.530078:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2489:1853163991579520:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316603.530717:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.530728:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.530735:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.530748:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cd500 x1853163991579392/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/800 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.530761:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 216 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1767316603.535574:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316603.535597:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991579648:192.168.204.145@tcp:400:kworker.0 00000002:00010000:0.0:1767316603.537064:0:46936:0:(mdc_locks.c:969:mdc_finish_enqueue()) ### DoM lock is returned by: getattr, size: 0 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000002:00010000:0.0:1767316603.537088:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316603.537146:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1070 (0 0) 00000080:00200000:0.0:1767316603.537155:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x42c:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1767316603.537271:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x42e:0x0](ffff88c5c85dd348): rc = 0 00000080:00200000:0.0:1767316603.537286:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x42e:0x0]: layout version change: 4294967294 -> 2 00000080:00010000:0.0:1767316603.537292:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42e:0x0](ffff88c5c85dd348) for lock 0x5295060e33d08aae 00000080:00010000:0.0:1767316603.537302:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42e:0x0](ffff88c5c85dd348) 00000080:00010000:0.0:1767316603.537306:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08aae from it ffffb10b019a7df8 00010000:00010000:0.0:1767316603.537310:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316603.537319:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316603.537365:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1767316603.537374:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=f130b.replay-single, dir=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) 00010000:00010000:0.0:1767316603.537404:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.537432:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.537482:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316603.537498:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x42e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:0.0:1767316603.537509:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x42e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316603.537518:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x42e:0x0]: losing layout lock 00010000:00010000:0.0:1767316603.537550:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316603.537557:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.537563:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1767316603.537566:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191aa00/0x5295060e33d08aae lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x42e:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67796e5 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316603.537581:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191be00/0x5295060e33d08a99 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67796a6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316603.537596:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316603.537608:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6402300 x1853163991580032/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316603.537631:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580032:192.168.204.145@tcp:36:rm.0 00000100:00100000:0.0:1767316603.537716:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:3.0:1767316603.547953:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2489:1853163991579520:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316603.548026:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991579648:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316603.554790:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991579776:192.168.204.145@tcp:9:kworker.0 00000100:00080000:1.0:1767316603.558526:0:2485:0:(import.c:1990:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff88c5c57d7800 x1853163991579776/t0(0) o9->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316613 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316603.558541:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1767316603.558545:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991579776:192.168.204.145@tcp:9:kworker.0 00000100:00100000:0.0:1767316603.559623:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580032:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316603.559692:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x42e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1767316603.559707:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00020000:00200000:0.0:1767316603.559749:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[1] flags 0x0 00000080:00200000:0.0:1767316603.559771:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x42e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316603.559780:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x42e:0x0](ffff88c5c85dd348) 00000080:00200008:0.0:1767316603.559861:0:46936:0:(file.c:424:ll_file_release()) START file d130b.replay-single:[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8), flags 704000 00010000:00010000:0.0:1767316603.559885:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.559929:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316603.559944:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.559952:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.559960:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.559966:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316603.559971:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316603.559974:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a92 00000080:00010000:0.0:1767316603.559980:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a92 from it ffffb10b019a7c10 00010000:00010000:0.0:1767316603.559984:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.559991:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.560005:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316603.560037:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d130b.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=0 00000080:00200000:0.0:1767316603.560233:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d130b.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:0.0:1767316603.560248:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d130b.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.560307:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.560315:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.560324:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316603.560347:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580160:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316603.560453:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.564812:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580160:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316603.564841:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.564858:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.564866:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.564877:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.564888:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.564919:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.564927:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.564934:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.564945:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6402a00 x1853163991580160/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316603.564958:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.564969:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1068 (0 0) 00000080:00200000:0.0:1767316603.564975:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1767316603.564995:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x42c:0x0], updating i_size 11776 00000080:00200000:0.0:1767316603.564999:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8): rc = 0 00000080:00010000:0.0:1767316603.565005:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) for lock 0x5295060e33d08ab5 00000080:00010000:0.0:1767316603.565015:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) 00000080:00010000:0.0:1767316603.565019:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ab5 from it ffffb10b019a7e00 00010000:00010000:0.0:1767316603.565022:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.565030:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.565086:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316603.565173:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.565183:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.565192:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.565198:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316603.565203:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316603.565206:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08a92 00000080:00010000:0.0:1767316603.565210:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a92 from it ffffb10b019a7d88 00010000:00010000:0.0:1767316603.565212:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.565219:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.565227:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1767316603.565234:0:46936:0:(namei.c:2264:ll_rmdir()) VFS Op:name=d130b.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:0.0:1767316603.565254:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.565287:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.565297:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.565349:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cce00 x1853163991579136/t51539607565(51539607565) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1767316603.565359:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1767316603.565364:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6402a00 x1853163991580416/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1767316603.565385:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580416:192.168.204.145@tcp:35:rm.0 00000100:00100000:0.0:1767316603.565449:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.570059:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580416:192.168.204.145@tcp:35:rm.0 00000002:00100000:0.0:1767316603.570131:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cce00 x1853163991579136/t51539607565(51539607565) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1767316603.570208:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.570217:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 1/0,0 mode: --/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.570224:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.570230:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:0.0:1767316603.570232:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08ab5 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677971d expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.570245:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08aa7 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x42c:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67796c2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.570293:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08a92 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677968a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316603.570314:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316603.570321:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6402a00 x1853163991580672/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 512/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316603.570342:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580672:192.168.204.145@tcp:36:rm.0 00000100:00100000:0.0:1767316603.570582:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.578243:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580672:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316603.578411:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x42c:0x0](ffff88c5c85cc2c8) 00000080:00200000:0.0:1767316603.578482:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316603.578502:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.578555:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.578564:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.578573:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316603.578593:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580800:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316603.578673:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.582858:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580800:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316603.582887:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.582916:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.582928:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.582939:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.582950:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.582956:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.582964:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.582974:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6400e00 x1853163991580800/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316603.582986:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.582999:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1767316603.583006:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.583219:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 11776 00000080:00010000:1.0:1767316603.583228:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08abc 00000080:00010000:1.0:1767316603.583234:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.583238:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08abc from it ffffb10b019a7b58 00010000:00010000:1.0:1767316603.583241:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.583249:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.583537:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.583559:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d134.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:1.0:1767316603.583563:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d134.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:1.0:1767316603.583589:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d134.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.583643:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.583650:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.583659:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.583689:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580928:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.583820:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.596076:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991580928:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.596143:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596168:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596176:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596187:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596199:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596212:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596220:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596227:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.596237:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cd180 x1853163991580928/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.596279:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596290:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1077 (0 0) 00000080:00200000:1.0:1767316603.596297:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316603.596320:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x435:0x0], updating i_size 11776 00000080:00200000:1.0:1767316603.596323:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x435:0x0](ffff88c5c85d3248): rc = 0 00000080:00010000:1.0:1767316603.596329:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) for lock 0x5295060e33d08ac3 00000080:00010000:1.0:1767316603.596341:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) 00000080:00010000:1.0:1767316603.596344:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ac3 from it ffffb10b019a7b20 00010000:00010000:1.0:1767316603.596347:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596556:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316603.596597:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d134.replay-single:[0x200000bd1:0x435:0x0](ffff88c5c85d3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.596603:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248),name=d134.replay-single 00000002:00010000:1.0:1767316603.596625:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x435:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.596643:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596652:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.596657:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.596662:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) 00000080:00010000:1.0:1767316603.596665:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) for lock 0x5295060e33d08ac3 00000080:00010000:1.0:1767316603.596674:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ac3 from it ffffb10b019a7cb8 00010000:00010000:1.0:1767316603.596677:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596684:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.596736:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316603.596743:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.596750:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596757:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.596762:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.596766:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.596768:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08abc 00000080:00010000:1.0:1767316603.596771:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08abc from it ffffb10b019a7bf8 00010000:00010000:1.0:1767316603.596774:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.596779:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.596786:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.596792:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d134.replay-single, flags=266 00000080:00200000:1.0:1767316603.596797:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 44 00000080:00200008:1.0:1767316603.596804:0:46936:0:(file.c:1001:ll_file_open()) START file d134.replay-single:[0x200000bd1:0x435:0x0](ffff88c5c85d3248), flags 704400 00000002:00010000:1.0:1767316603.596824:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x435:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100240704401 00010000:00010000:1.0:1767316603.596873:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.596881:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:1.0:1767316603.596892:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316603.596900:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cd180 x1853163991581184/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316603.596910:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.596931:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581184:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.597089:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.604557:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581184:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.604584:0:46936:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.604596:0:46936:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.604604:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.604611:0:46936:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.604617:0:46936:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.604650:0:46936:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5d191ae00 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:1.0:1767316603.604654:0:46936:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.604661:0:46936:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.604669:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.604675:0:46936:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1767316603.604682:0:46936:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.604688:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08aca lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00000002:00100000:1.0:1767316603.604707:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c65cd180 x1853163991581184/t51539607569(51539607569) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316603.604721:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cd180 x1853163991581184/t51539607569(51539607569) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316603.604734:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.604754:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x435:0x0], updating i_size 11776 00000002:00100000:1.0:1767316603.604771:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cd180 x1853163991581184/t51539607569(51539607569) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:1.0:1767316603.604835:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d134.replay-single:[0x200000bd1:0x435:0x0](ffff88c5c85d3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.604842:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248),name=d134.replay-single 00000002:00010000:1.0:1767316603.604863:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x435:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.604883:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.604893:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.604900:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.604905:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) 00000080:00010000:1.0:1767316603.604909:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) for lock 0x5295060e33d08ac3 00000080:00010000:1.0:1767316603.604914:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ac3 from it ffffb10b019a7cd8 00010000:00010000:1.0:1767316603.604917:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.604925:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.604955:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248) pos/size0/11776 32bit_api 0 00000002:00010000:1.0:1767316603.604967:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x435:0x0]) in obj [0x200000bd1:0x435:0x0], intent: readdir flags 0 00010000:00010000:1.0:1767316603.604986:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.604994:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316603.605701:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581312:192.168.204.145@tcp:37:rm.0 00000100:00100000:1.0:1767316603.605801:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.615618:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581312:192.168.204.145@tcp:37:rm.0 00010000:00010000:1.0:1767316603.616329:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.616359:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316603.616417:0:46936:0:(file.c:424:ll_file_release()) START file d134.replay-single:[0x200000bd1:0x435:0x0](ffff88c5c85d3248), flags 704000 00000002:00100000:1.0:1767316603.616476:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cd180 x1853163991581184/t51539607569(51539607569) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1767316603.616488:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1767316603.616492:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65ced80 x1853163991581568/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1767316603.616516:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581568:192.168.204.145@tcp:35:rm.0 00000100:00100000:1.0:1767316603.616594:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.619826:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581568:192.168.204.145@tcp:35:rm.0 00000002:00100000:1.0:1767316603.619859:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cd180 x1853163991581184/t51539607569(51539607569) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316603.619936:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316603.619954:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.619970:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.619982:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.619989:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.619994:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:1.0:1767316603.619997:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08abc 00000080:00010000:1.0:1767316603.620020:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08abc from it ffffb10b019a7bf8 00010000:00010000:1.0:1767316603.620024:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.620031:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.620274:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.620283:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d134.replay-single, flags=266 00000080:00200000:1.0:1767316603.620289:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 44 00000080:00200008:1.0:1767316603.620297:0:46936:0:(file.c:1001:ll_file_open()) START file d134.replay-single:[0x200000bd1:0x435:0x0](ffff88c5c85d3248), flags 704400 00000002:00010000:1.0:1767316603.620313:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x435:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0104240704401 00010000:00010000:1.0:1767316603.620362:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.620370:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:1.0:1767316603.620381:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316603.620387:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65ced80 x1853163991581824/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316603.620396:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.620415:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581824:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.620580:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.625777:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581824:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.625808:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.625835:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.625846:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.625857:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.625873:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=200002b, status=0 req@ffff88c5c65ced80 x1853163991581824/t51539607571(51539607571) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316603.625886:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65ced80 x1853163991581824/t51539607571(51539607571) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.625896:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.625907:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 16 res 8589937617/1077 (0 0) 00000080:00200000:1.0:1767316603.625913:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.625934:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x435:0x0], updating i_size 11776 00000080:00010000:1.0:1767316603.625939:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) for lock 0x5295060e33d08ad1 00000080:00010000:1.0:1767316603.625946:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ad1 from it ffffb10b019a7bd0 00010000:00010000:1.0:1767316603.625949:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.625958:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.625994:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65ced80 x1853163991581824/t51539607571(51539607571) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:1.0:1767316603.626056:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d134.replay-single:[0x200000bd1:0x435:0x0](ffff88c5c85d3248), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.626063:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248),name=d134.replay-single 00000002:00010000:1.0:1767316603.626082:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x435:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.626091:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.626137:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.626144:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.626149:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) 00000080:00010000:1.0:1767316603.626153:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) for lock 0x5295060e33d08ac3 00000080:00010000:1.0:1767316603.626157:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ac3 from it ffffb10b019a7cd8 00010000:00010000:1.0:1767316603.626160:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.626167:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.626198:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248) pos/size0/11776 32bit_api 0 00000080:00200000:1.0:1767316603.626222:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248) pos/size18446744073709551614/11776 32bit_api 0 00000080:00200000:1.0:1767316603.626241:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.626255:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=A, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), flags=0 00000080:00200000:1.0:1767316603.626265:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=A, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), intent=getattr 00000002:00010000:1.0:1767316603.626281:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: A,[0x0:0x0:0x0]) in obj [0x200000bd1:0x435:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.626323:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.626330:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.626338:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.626356:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581952:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.626431:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.633400:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991581952:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.633429:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.633450:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.633458:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.633469:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.633479:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.633489:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.633514:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.633520:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.633531:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cd180 x1853163991581952/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/584 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.633542:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.633552:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1078 (0 0) 00000080:00200000:1.0:1767316603.633558:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x435:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316603.633579:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x436:0x0], updating i_size 11776 00000080:00200000:1.0:1767316603.633582:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x436:0x0](ffff88c5c85da1c8): rc = 0 00000080:00010000:1.0:1767316603.633588:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) for lock 0x5295060e33d08ad8 00000080:00010000:1.0:1767316603.633600:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) 00000080:00010000:1.0:1767316603.633603:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ad8 from it ffffb10b019a7b20 00010000:00010000:1.0:1767316603.633606:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.633614:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316603.633658:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file A:[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.634194:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8),name=A 00000002:00010000:1.0:1767316603.634212:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x436:0x0]) in obj [0x200000bd1:0x435:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.634228:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.634238:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.634245:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.634251:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) 00000080:00010000:1.0:1767316603.634254:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) for lock 0x5295060e33d08ad8 00000080:00010000:1.0:1767316603.634268:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ad8 from it ffffb10b019a7cb8 00010000:00010000:1.0:1767316603.634271:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.634279:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.634325:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.634332:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=A, flags=258 00000080:00200000:1.0:1767316603.634340:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), inode mode 41ed mask 44 00000080:00200008:1.0:1767316603.634348:0:46936:0:(file.c:1001:ll_file_open()) START file A:[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), flags 704400 00000002:00010000:1.0:1767316603.634361:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x436:0x0]) in obj [0x200000bd1:0x435:0x0], intent: open flags 0100240704401 00010000:00010000:1.0:1767316603.634402:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.634409:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:1.0:1767316603.634419:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316603.634426:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cd180 x1853163991582208/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316603.634447:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.634466:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991582208:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.634628:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00010000:00010000:0.0:1767316603.634806:0:46937:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x10000000000000 pid: 46937 initiator: MDT0 00010000:00010000:0.0:1767316603.634813:0:46937:0:(ldlm_request.c:1177:ldlm_cli_lock_create_pack()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT flags: 0x0 pid: 46937 initiator: MDT0 00000100:00100000:1.0:1767316603.635532:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991582336:192.168.204.145@tcp:63:ll_sa.0 00000100:00100000:1.0:1767316603.641822:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991582208:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.641844:0:46936:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.641857:0:46936:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.641863:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.641870:0:46936:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.641876:0:46936:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.641882:0:46936:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5d191a400 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:1.0:1767316603.641886:0:46936:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 5 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.641892:0:46936:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 5 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.641898:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 5/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 5 type: IBT flags: 0x20e400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.641918:0:46936:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1767316603.641924:0:46936:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.641929:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08adf lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 5 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00000002:00100000:1.0:1767316603.641951:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c65cd180 x1853163991582208/t51539607572(51539607572) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316603.641964:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cd180 x1853163991582208/t51539607572(51539607572) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316603.641975:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.641995:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x436:0x0], updating i_size 11776 00000002:00100000:1.0:1767316603.642010:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cd180 x1853163991582208/t51539607572(51539607572) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:1.0:1767316603.642082:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file A:[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.642088:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8),name=A 00000002:00010000:1.0:1767316603.642149:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x436:0x0]) in obj [0x200000bd1:0x435:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.642164:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.642176:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.642182:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.642187:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) 00000080:00010000:1.0:1767316603.642190:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) for lock 0x5295060e33d08ad8 00000080:00010000:1.0:1767316603.642201:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ad8 from it ffffb10b019a7cd8 00010000:00010000:1.0:1767316603.642205:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.642212:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.642243:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8) pos/size0/11776 32bit_api 0 00000002:00010000:1.0:1767316603.642255:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x436:0x0]) in obj [0x200000bd1:0x436:0x0], intent: readdir flags 0 00010000:00010000:1.0:1767316603.642273:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.642281:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316603.642896:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991582464:192.168.204.145@tcp:37:rm.0 00000100:00100000:1.0:1767316603.642977:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.652344:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991582464:192.168.204.145@tcp:37:rm.0 00010000:00010000:1.0:1767316603.653013:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.653027:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316603.653087:0:46936:0:(file.c:424:ll_file_release()) START file A:[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), flags 704000 00000002:00100000:1.0:1767316603.653693:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cd180 x1853163991582208/t51539607572(51539607572) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1767316603.653706:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1767316603.653727:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cc000 x1853163991582720/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1767316603.653749:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991582720:192.168.204.145@tcp:35:rm.0 00000100:00100000:1.0:1767316603.653889:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00010000:00010000:1.0:1767316603.655285:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316603.661680:0:2486:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.661710:0:2486:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.661719:0:2486:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.661729:0:2486:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.661740:0:2486:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.661751:0:2486:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.661759:0:2486:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.661765:0:2486:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.661780:0:2486:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6401f80 x1853163991582336/t0(0) o63->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:24/4 lens 664/624 e 0 to 0 dl 1767316619 ref 1 fl Interpret:RQU/200/0 rc 0/0 job:'ll_sa.0' uid:0 gid:0 projid:4294967295 00000080:00010000:0.0:1767316603.661791:0:2486:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ae6 from it ffff88c5c5679988 00010000:00010000:0.0:1767316603.661794:0:2486:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.661812:0:2486:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1767316603.661842:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991582336:192.168.204.145@tcp:63:ll_sa.0 00010000:00010000:0.0:1767316603.661874:0:44224:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.661893:0:44224:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x438:0x0], updating i_size 11776 00000080:00200000:0.0:1767316603.661896:0:44224:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x438:0x0](ffff88c5c85d9988): rc = 0 00000080:00010000:0.0:1767316603.661901:0:44224:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) for lock 0x5295060e33d08ae6 00000080:00010000:0.0:1767316603.661906:0:44224:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ae6 from it ffff88c5c5679988 00010000:00010000:0.0:1767316603.661909:0:44224:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.661916:0:44224:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316603.661990:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991582720:192.168.204.145@tcp:35:rm.0 00000002:00100000:1.0:1767316603.662011:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cd180 x1853163991582208/t51539607572(51539607572) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316603.662193:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.662202:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=A, flags=258 00000080:00200000:1.0:1767316603.662209:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), inode mode 41ed mask 44 00000080:00200008:1.0:1767316603.662218:0:46936:0:(file.c:1001:ll_file_open()) START file A:[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), flags 704400 00000002:00010000:1.0:1767316603.662237:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x436:0x0]) in obj [0x200000bd1:0x435:0x0], intent: open flags 0104240704401 00010000:00010000:1.0:1767316603.662434:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.662442:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:1.0:1767316603.662454:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316603.662461:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cc000 x1853163991582976/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316603.662469:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.662489:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991582976:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.662703:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.666769:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991582976:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.666802:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.666811:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.666821:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.666831:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.666845:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=200002b, status=0 req@ffff88c5c65cc000 x1853163991582976/t51539607574(51539607574) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316603.666858:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cc000 x1853163991582976/t51539607574(51539607574) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.666868:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.666878:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 16 res 8589937617/1078 (0 0) 00000080:00200000:1.0:1767316603.666885:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.666904:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x436:0x0], updating i_size 11776 00000080:00010000:1.0:1767316603.666909:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) for lock 0x5295060e33d08aed 00000080:00010000:1.0:1767316603.666916:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08aed from it ffffb10b019a7bd0 00010000:00010000:1.0:1767316603.666920:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.666947:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.666972:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cc000 x1853163991582976/t51539607574(51539607574) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:1.0:1767316603.667038:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file A:[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.667453:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8),name=A 00000002:00010000:1.0:1767316603.667474:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x436:0x0]) in obj [0x200000bd1:0x435:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.667484:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.667494:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.667500:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.667506:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) 00000080:00010000:1.0:1767316603.667509:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) for lock 0x5295060e33d08ad8 00000080:00010000:1.0:1767316603.667513:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ad8 from it ffffb10b019a7cd8 00010000:00010000:1.0:1767316603.667516:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.667522:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.667555:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8) pos/size0/11776 32bit_api 0 00000080:00200000:1.0:1767316603.667577:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8) pos/size18446744073709551614/11776 32bit_api 0 00000080:00200000:1.0:1767316603.667597:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.667618:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=f134.replay-single, dir=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), flags=0 00000080:00200000:1.0:1767316603.667622:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f134.replay-single, dir=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), intent=getattr 00000002:00010000:1.0:1767316603.667642:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f134.replay-single,[0x0:0x0:0x0]) in obj [0x200000bd1:0x436:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.667691:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.667698:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.667707:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.667730:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991583104:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.667853:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.671591:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991583104:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.671616:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.671647:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.671655:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.671665:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.671676:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.671707:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.671714:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.671724:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.671734:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cd180 x1853163991583104/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/656 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.671746:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 72 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316603.671759:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.671769:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1082 (0 0) 00000080:00200000:1.0:1767316603.671776:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x436:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316603.671845:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x43a:0x0](ffff88c5c85c80c8): rc = 0 00000080:00200000:1.0:1767316603.671858:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x43a:0x0]: layout version change: 4294967294 -> 65535 00000080:00010000:1.0:1767316603.671863:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43a:0x0](ffff88c5c85c80c8) for lock 0x5295060e33d08af4 00000080:00010000:1.0:1767316603.671872:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x43a:0x0](ffff88c5c85c80c8) 00000080:00010000:1.0:1767316603.671875:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08af4 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316603.671878:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.671896:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316603.671925:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), inode mode 41ed mask 3 00000080:00200000:1.0:1767316603.671933:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=f134.replay-single, dir=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8) 00010000:00010000:1.0:1767316603.671961:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.671978:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.672063:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316603.672075:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x43a:0x0]: losing layout lock 00010000:00010000:1.0:1767316603.672154:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.672162:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.672166:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1767316603.672169:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08af4 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x43a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67797f6 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.672183:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ad8 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677979b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:1.0:1767316603.672195:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316603.672208:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cd180 x1853163991583360/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316603.672229:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991583360:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316603.672484:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.679597:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991583360:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316603.679659:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x43a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316603.679671:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316603.679721:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x43a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316603.679728:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x43a:0x0](ffff88c5c85c80c8) 00000080:00200008:1.0:1767316603.679800:0:46936:0:(file.c:424:ll_file_release()) START file A:[0x200000bd1:0x436:0x0](ffff88c5c85da1c8), flags 704000 00010000:00010000:1.0:1767316603.679822:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.679855:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.679869:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=A, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), flags=0 00000080:00200000:1.0:1767316603.679873:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=A, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), intent=getattr 00000002:00010000:1.0:1767316603.679909:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: A,[0x0:0x0:0x0]) in obj [0x200000bd1:0x435:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.679956:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.679964:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.679973:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.679992:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991583488:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.680057:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.684930:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991583488:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.684957:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.684974:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.684982:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.684994:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.685004:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.685015:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.685021:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.685028:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.685050:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65ce680 x1853163991583488/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/584 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.685062:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.685090:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1078 (0 0) 00000080:00200000:1.0:1767316603.685130:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x435:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316603.685151:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x436:0x0], updating i_size 11776 00000080:00200000:1.0:1767316603.685154:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x436:0x0](ffff88c5c85da1c8): rc = 0 00000080:00010000:1.0:1767316603.685160:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) for lock 0x5295060e33d08afb 00000080:00010000:1.0:1767316603.685171:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x436:0x0](ffff88c5c85da1c8) 00000080:00010000:1.0:1767316603.685174:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08afb from it ffffb10b019a7e00 00010000:00010000:1.0:1767316603.685178:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.685185:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.685213:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 3 00000080:00200000:1.0:1767316603.685222:0:46936:0:(namei.c:2264:ll_rmdir()) VFS Op:name=A, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248) 00010000:00010000:1.0:1767316603.685246:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.685274:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.685292:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.685302:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.685338:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cc000 x1853163991582976/t51539607574(51539607574) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1767316603.685364:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1767316603.685371:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65ce680 x1853163991583744/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1767316603.685393:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991583744:192.168.204.145@tcp:35:rm.0 00000100:00100000:1.0:1767316603.685467:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.689183:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991583744:192.168.204.145@tcp:35:rm.0 00000002:00100000:1.0:1767316603.689218:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cc000 x1853163991582976/t51539607574(51539607574) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:1.0:1767316603.689304:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.689313:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 1/0,0 mode: --/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.689320:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.689326:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:1.0:1767316603.689329:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08afb lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x436:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677982e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.689341:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08aed lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x436:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67797d3 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.689352:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08ac3 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677975c expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:1.0:1767316603.689383:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316603.689389:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65ce680 x1853163991584000/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 496/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316603.689412:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584000:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316603.689481:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.699088:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584000:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316603.699175:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x436:0x0](ffff88c5c85da1c8) 00000080:00200000:1.0:1767316603.699237:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.699253:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=B, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), flags=0 00000080:00200000:1.0:1767316603.699271:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=B, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), intent=getattr 00010000:00010000:1.0:1767316603.699289:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1767316603.699306:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ae6 from it ffffb10b019a7910 00010000:00010000:1.0:1767316603.699309:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.699317:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316603.699343:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file B:[0x200000bd1:0x438:0x0](ffff88c5c85d9988), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.699349:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988),name=B 00000002:00010000:1.0:1767316603.699366:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x438:0x0]) in obj [0x200000bd1:0x435:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.699376:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.699404:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.699411:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.699417:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) 00000080:00010000:1.0:1767316603.699420:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) for lock 0x5295060e33d08ae6 00000080:00010000:1.0:1767316603.699426:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ae6 from it ffffb10b019a7cb8 00010000:00010000:1.0:1767316603.699429:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.699435:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.699464:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.699470:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=B, flags=258 00000080:00200000:1.0:1767316603.699475:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988), inode mode 41ed mask 44 00000080:00200008:1.0:1767316603.699482:0:46936:0:(file.c:1001:ll_file_open()) START file B:[0x200000bd1:0x438:0x0](ffff88c5c85d9988), flags 704400 00000002:00010000:1.0:1767316603.699496:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x438:0x0]) in obj [0x200000bd1:0x435:0x0], intent: open flags 0100240704401 00010000:00010000:1.0:1767316603.699543:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.699550:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:1.0:1767316603.699560:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316603.699567:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cc000 x1853163991584256/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316603.699576:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.699603:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584256:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.699769:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.704775:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584256:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.704803:0:46936:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.704817:0:46936:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.704824:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.704831:0:46936:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.704838:0:46936:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.704844:0:46936:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5d191b800 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:1.0:1767316603.704849:0:46936:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.704856:0:46936:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.704863:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 5/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.704871:0:46936:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1767316603.704879:0:46936:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.704902:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b02 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00000002:00100000:1.0:1767316603.704921:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c65cc000 x1853163991584256/t51539607578(51539607578) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316603.704934:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cc000 x1853163991584256/t51539607578(51539607578) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316603.704947:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.704969:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x438:0x0], updating i_size 11776 00000002:00100000:1.0:1767316603.704986:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cc000 x1853163991584256/t51539607578(51539607578) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:1.0:1767316603.705067:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file B:[0x200000bd1:0x438:0x0](ffff88c5c85d9988), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.705075:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988),name=B 00000002:00010000:1.0:1767316603.705126:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x438:0x0]) in obj [0x200000bd1:0x435:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.705140:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.705152:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.705160:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.705165:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) 00000080:00010000:1.0:1767316603.705169:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) for lock 0x5295060e33d08ae6 00000080:00010000:1.0:1767316603.705175:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ae6 from it ffffb10b019a7cd8 00010000:00010000:1.0:1767316603.705178:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.705199:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.705232:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988) pos/size0/11776 32bit_api 0 00000002:00010000:1.0:1767316603.705244:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x438:0x0]) in obj [0x200000bd1:0x438:0x0], intent: readdir flags 0 00010000:00010000:1.0:1767316603.705250:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.705269:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1767316603.705893:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584384:192.168.204.145@tcp:37:rm.0 00000100:00100000:1.0:1767316603.705986:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.719913:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584384:192.168.204.145@tcp:37:rm.0 00010000:00010000:1.0:1767316603.720644:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.720659:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00200008:1.0:1767316603.720721:0:46936:0:(file.c:424:ll_file_release()) START file B:[0x200000bd1:0x438:0x0](ffff88c5c85d9988), flags 704000 00000002:00100000:1.0:1767316603.720783:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cc000 x1853163991584256/t51539607578(51539607578) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:1.0:1767316603.720795:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1767316603.720800:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cdf80 x1853163991584640/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:1.0:1767316603.720842:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584640:192.168.204.145@tcp:35:rm.0 00000100:00100000:1.0:1767316603.720925:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.724849:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584640:192.168.204.145@tcp:35:rm.0 00000002:00100000:1.0:1767316603.724884:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cc000 x1853163991584256/t51539607578(51539607578) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:1.0:1767316603.724956:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.724965:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=B, flags=258 00000080:00200000:1.0:1767316603.724971:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988), inode mode 41ed mask 44 00000080:00200008:1.0:1767316603.724980:0:46936:0:(file.c:1001:ll_file_open()) START file B:[0x200000bd1:0x438:0x0](ffff88c5c85d9988), flags 704400 00000002:00010000:1.0:1767316603.725002:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x438:0x0]) in obj [0x200000bd1:0x435:0x0], intent: open flags 0104240704401 00010000:00010000:1.0:1767316603.725217:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.725227:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:1.0:1767316603.725239:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:1.0:1767316603.725246:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cdf80 x1853163991584896/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:1.0:1767316603.725457:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.725481:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584896:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.725706:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.732824:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991584896:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.732865:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.732892:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.732904:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.732915:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.732931:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=200002b, status=0 req@ffff88c5c65cdf80 x1853163991584896/t51539607580(51539607580) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:1.0:1767316603.732944:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cdf80 x1853163991584896/t51539607580(51539607580) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.732955:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.732965:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 16 res 8589937617/1080 (0 0) 00000080:00200000:1.0:1767316603.732972:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 0 00000080:00200000:1.0:1767316603.732992:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x438:0x0], updating i_size 11776 00000080:00010000:1.0:1767316603.732997:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) for lock 0x5295060e33d08b09 00000080:00010000:1.0:1767316603.733006:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b09 from it ffffb10b019a7bd0 00010000:00010000:1.0:1767316603.733010:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.733018:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.733433:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c65cdf80 x1853163991584896/t51539607580(51539607580) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:1.0:1767316603.733500:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file B:[0x200000bd1:0x438:0x0](ffff88c5c85d9988), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316603.733507:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988),name=B 00000002:00010000:1.0:1767316603.733529:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x438:0x0]) in obj [0x200000bd1:0x435:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.733539:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.733550:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.733556:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x435:0x0] suppgids 0 0: rc 1 00000080:00010000:1.0:1767316603.733561:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) 00000080:00010000:1.0:1767316603.733564:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) for lock 0x5295060e33d08ae6 00000080:00010000:1.0:1767316603.733569:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ae6 from it ffffb10b019a7cd8 00010000:00010000:1.0:1767316603.733572:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.733579:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.733611:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988) pos/size0/11776 32bit_api 0 00000080:00200000:1.0:1767316603.733644:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988) pos/size18446744073709551614/11776 32bit_api 0 00000080:00200000:1.0:1767316603.733665:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.733679:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=f134.replay-single, dir=[0x200000bd1:0x438:0x0](ffff88c5c85d9988), flags=0 00000080:00200000:1.0:1767316603.733683:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f134.replay-single, dir=[0x200000bd1:0x438:0x0](ffff88c5c85d9988), intent=getattr 00000002:00010000:1.0:1767316603.733701:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f134.replay-single,[0x0:0x0:0x0]) in obj [0x200000bd1:0x438:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.733747:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.733754:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.733763:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.733784:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585024:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.733911:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.740441:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585024:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.740468:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740488:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740496:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740506:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740517:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740528:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740553:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740560:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.740572:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cc000 x1853163991585024/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/736 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.740581:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 152 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316603.740594:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.740603:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1083 (0 0) 00000080:00200000:1.0:1767316603.740609:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x438:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316603.740661:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x43b:0x0](ffff88c5c85d3a88): rc = 0 00000080:00200000:1.0:1767316603.740674:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x43b:0x0]: layout version change: 4294967294 -> 1 00000080:00010000:1.0:1767316603.740679:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43b:0x0](ffff88c5c85d3a88) for lock 0x5295060e33d08b10 00000080:00010000:1.0:1767316603.740689:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x43b:0x0](ffff88c5c85d3a88) 00000080:00010000:1.0:1767316603.740692:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b10 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316603.740695:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.740703:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316603.740753:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988), inode mode 41ed mask 3 00000080:00200000:1.0:1767316603.740762:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=f134.replay-single, dir=[0x200000bd1:0x438:0x0](ffff88c5c85d9988) 00010000:00010000:1.0:1767316603.740792:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740809:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740873:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316603.740885:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x43b:0x0]: losing layout lock 00010000:00010000:1.0:1767316603.740913:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.740922:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.740927:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:1.0:1767316603.740930:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b10 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x43b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779882 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.740944:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08ae6 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67797c5 expref: -99 pid: 46937 timeout: 0 lvb_type: 0 00000020:00100000:1.0:1767316603.740959:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316603.740965:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cc000 x1853163991585280/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316603.740998:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585280:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316603.741083:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.752397:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585280:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316603.752459:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x43b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316603.752470:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316603.752492:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x43b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316603.752498:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x43b:0x0](ffff88c5c85d3a88) 00000080:00200008:1.0:1767316603.752569:0:46936:0:(file.c:424:ll_file_release()) START file B:[0x200000bd1:0x438:0x0](ffff88c5c85d9988), flags 704000 00010000:00010000:1.0:1767316603.752588:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.752616:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.752641:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=B, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), flags=0 00000080:00200000:1.0:1767316603.752645:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=B, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), intent=getattr 00000002:00010000:1.0:1767316603.752678:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: B,[0x0:0x0:0x0]) in obj [0x200000bd1:0x435:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.752953:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.752962:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.752972:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.752989:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585408:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.753687:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.761705:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585408:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.761749:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.761766:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.761773:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.761785:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.761796:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.761950:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.761958:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.761963:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.761975:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cf800 x1853163991585408/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/584 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.761986:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.762007:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1080 (0 0) 00000080:00200000:1.0:1767316603.762014:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x435:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316603.762034:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x438:0x0], updating i_size 11776 00000080:00200000:1.0:1767316603.762168:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x438:0x0](ffff88c5c85d9988): rc = 0 00000080:00010000:1.0:1767316603.762175:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) for lock 0x5295060e33d08b17 00000080:00010000:1.0:1767316603.762187:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x438:0x0](ffff88c5c85d9988) 00000080:00010000:1.0:1767316603.762190:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b17 from it ffffb10b019a7e00 00010000:00010000:1.0:1767316603.762193:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.762225:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1767316603.762266:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248), inode mode 41ed mask 3 00000080:00200000:1.0:1767316603.762275:0:46936:0:(namei.c:2264:ll_rmdir()) VFS Op:name=B, dir=[0x200000bd1:0x435:0x0](ffff88c5c85d3248) 00010000:00010000:1.0:1767316603.762307:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.762329:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.762594:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65cdf80 x1853163991584896/t51539607580(51539607580) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1767316603.762605:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1767316603.762611:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6401f80 x1853163991585664/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1767316603.762645:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585664:192.168.204.145@tcp:35:rm.0 00000100:00100000:0.0:1767316603.762741:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.766340:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585664:192.168.204.145@tcp:35:rm.0 00000002:00100000:0.0:1767316603.766390:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65cdf80 x1853163991584896/t51539607580(51539607580) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1767316603.766468:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.766476:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 1/0,0 mode: --/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.766481:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1767316603.766484:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a400/0x5295060e33d08b17 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x438:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67798ba expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.766495:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b800/0x5295060e33d08b09 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x438:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677985f expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316603.766515:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316603.766521:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6401f80 x1853163991585920/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316603.766544:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585920:192.168.204.145@tcp:36:rm.0 00000100:00100000:0.0:1767316603.766620:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.772271:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991585920:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316603.772314:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x438:0x0](ffff88c5c85d9988) 00000080:00200008:0.0:1767316603.772378:0:46936:0:(file.c:424:ll_file_release()) START file d134.replay-single:[0x200000bd1:0x435:0x0](ffff88c5c85d3248), flags 704000 00010000:00010000:0.0:1767316603.772409:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.772446:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316603.772475:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.772484:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.772492:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.772498:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316603.772503:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316603.772506:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08abc 00000080:00010000:0.0:1767316603.772512:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08abc from it ffffb10b019a7c10 00010000:00010000:0.0:1767316603.772516:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.772522:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.772537:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316603.772556:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d134.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=0 00000080:00200000:0.0:1767316603.772560:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d134.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:0.0:1767316603.772571:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d134.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.772610:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.772617:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.772625:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316603.772656:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586048:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316603.772728:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.786189:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586048:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316603.786220:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786239:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786246:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786259:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786270:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786282:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786290:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786298:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.786309:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6403800 x1853163991586048/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316603.786322:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786348:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1077 (0 0) 00000080:00200000:0.0:1767316603.786357:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1767316603.786382:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x435:0x0], updating i_size 11776 00000080:00200000:0.0:1767316603.786385:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x435:0x0](ffff88c5c85d3248): rc = 0 00000080:00010000:0.0:1767316603.786392:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) for lock 0x5295060e33d08b1e 00000080:00010000:0.0:1767316603.786403:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x435:0x0](ffff88c5c85d3248) 00000080:00010000:0.0:1767316603.786407:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b1e from it ffffb10b019a7e00 00010000:00010000:0.0:1767316603.786410:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786418:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.786451:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316603.786468:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.786476:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786486:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.786491:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316603.786496:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316603.786499:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08abc 00000080:00010000:0.0:1767316603.786503:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08abc from it ffffb10b019a7d88 00010000:00010000:0.0:1767316603.786505:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786523:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.786532:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1767316603.786540:0:46936:0:(namei.c:2264:ll_rmdir()) VFS Op:name=d134.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:0.0:1767316603.786563:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786587:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.786599:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.786653:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c65ced80 x1853163991581824/t51539607571(51539607571) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1767316603.786664:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1767316603.786669:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6403800 x1853163991586304/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1767316603.786693:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586304:192.168.204.145@tcp:35:rm.0 00000100:00100000:0.0:1767316603.786875:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.790751:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586304:192.168.204.145@tcp:35:rm.0 00000002:00100000:0.0:1767316603.790789:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c65ced80 x1853163991581824/t51539607571(51539607571) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1767316603.790873:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.790894:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 1/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.790902:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.790907:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:0.0:1767316603.790911:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b1e lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x435:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a67798eb expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.790924:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ae00/0x5295060e33d08ad1 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x435:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779778 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.790938:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08abc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779740 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316603.790958:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316603.790965:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6403800 x1853163991586560/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 512/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316603.790986:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586560:192.168.204.145@tcp:36:rm.0 00000100:00100000:0.0:1767316603.791064:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.796448:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586560:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316603.796494:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x435:0x0](ffff88c5c85d3248) 00000080:00200000:0.0:1767316603.796565:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316603.796582:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.796650:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.796671:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.796682:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316603.796703:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586688:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316603.796770:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.802909:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586688:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316603.802934:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.802945:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.802955:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.802966:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.802977:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.802985:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.802992:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.803023:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6402680 x1853163991586688/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316603.803159:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.803175:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1767316603.803183:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316603.803216:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 11776 00000080:00010000:0.0:1767316603.803223:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08b25 00000080:00010000:0.0:1767316603.803229:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316603.803231:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b25 from it ffffb10b019a7b58 00010000:00010000:0.0:1767316603.803235:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.803244:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.804248:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316603.804288:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d135.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:0.0:1767316603.804293:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d135.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:0.0:1767316603.804318:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d135.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.804539:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.804549:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.804558:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316603.804592:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586816:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316603.804784:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.815825:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991586816:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316603.815852:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.815874:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.815882:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.815891:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.815901:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.815912:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.815919:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.815925:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.815936:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6402680 x1853163991586816/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316603.815961:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.815973:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1084 (0 0) 00000080:00200000:0.0:1767316603.815981:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1767316603.816002:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x43c:0x0], updating i_size 12800 00000080:00200000:0.0:1767316603.816007:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x43c:0x0](ffff88c5c85d0908): rc = 0 00000080:00010000:0.0:1767316603.816012:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) for lock 0x5295060e33d08b2c 00000080:00010000:0.0:1767316603.816024:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00000080:00010000:0.0:1767316603.816027:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b2c from it ffffb10b019a7b20 00010000:00010000:0.0:1767316603.816030:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.816047:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200008:0.0:1767316603.816092:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d135.replay-single:[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1767316603.816135:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908),name=d135.replay-single 00000002:00010000:0.0:1767316603.816159:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x43c:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.816167:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.816176:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.816183:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316603.816187:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00000080:00010000:0.0:1767316603.816191:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) for lock 0x5295060e33d08b2c 00000080:00010000:0.0:1767316603.816202:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b2c from it ffffb10b019a7cb8 00010000:00010000:0.0:1767316603.816205:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.816213:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.816280:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316603.816287:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.816294:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.816302:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.816308:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316603.816312:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316603.816314:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08b25 00000080:00010000:0.0:1767316603.816317:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b25 from it ffffb10b019a7bf8 00010000:00010000:0.0:1767316603.816320:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.816327:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.816335:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316603.816341:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d135.replay-single, flags=266 00000080:00200000:0.0:1767316603.816346:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 44 00000080:00200008:0.0:1767316603.816354:0:46936:0:(file.c:1001:ll_file_open()) START file d135.replay-single:[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags 704400 00000002:00010000:0.0:1767316603.816375:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x43c:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100240704401 00010000:00010000:0.0:1767316603.816424:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.816430:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:0.0:1767316603.816440:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1767316603.816445:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6402680 x1853163991587072/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1767316603.816454:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316603.816474:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587072:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316603.816655:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.820900:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587072:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316603.820928:0:46936:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.820941:0:46936:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.820947:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.820954:0:46936:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.820961:0:46936:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.820986:0:46936:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c341cc00 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:0.0:1767316603.820991:0:46936:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.820997:0:46936:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.821005:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.821012:0:46936:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1767316603.821018:0:46936:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.821024:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b33 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00000002:00100000:0.0:1767316603.821056:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c6402680 x1853163991587072/t51539607586(51539607586) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1767316603.821068:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6402680 x1853163991587072/t51539607586(51539607586) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1767316603.821080:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316603.821142:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x43c:0x0], updating i_size 12800 00000002:00100000:0.0:1767316603.821158:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6402680 x1853163991587072/t51539607586(51539607586) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:0.0:1767316603.821221:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d135.replay-single:[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1767316603.821228:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908),name=d135.replay-single 00000002:00010000:0.0:1767316603.821248:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x43c:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.821278:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.821288:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.821295:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316603.821301:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00000080:00010000:0.0:1767316603.821304:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) for lock 0x5295060e33d08b2c 00000080:00010000:0.0:1767316603.821309:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b2c from it ffffb10b019a7cd8 00010000:00010000:0.0:1767316603.821312:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.821319:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316603.821351:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) pos/size0/12800 32bit_api 0 00000002:00010000:0.0:1767316603.821363:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x43c:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: readdir flags 0 00010000:00010000:0.0:1767316603.821369:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.821377:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1767316603.821990:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587200:192.168.204.145@tcp:37:rm.0 00000100:00100000:0.0:1767316603.822568:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.833688:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587200:192.168.204.145@tcp:37:rm.0 00010000:00010000:0.0:1767316603.834518:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.834552:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200008:2.0:1767316603.834775:0:46936:0:(file.c:424:ll_file_release()) START file d135.replay-single:[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags 704000 00000002:00100000:2.0:1767316603.834836:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c6402680 x1853163991587072/t51539607586(51539607586) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1767316603.834849:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1767316603.834855:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d5500 x1853163991587456/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1767316603.834880:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587456:192.168.204.145@tcp:35:rm.0 00000100:00100000:2.0:1767316603.834972:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.839422:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587456:192.168.204.145@tcp:35:rm.0 00000002:00100000:2.0:1767316603.839456:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c6402680 x1853163991587072/t51539607586(51539607586) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:2.0:1767316603.839544:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316603.839560:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.839578:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.839609:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.839616:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316603.839622:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316603.839625:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08b25 00000080:00010000:2.0:1767316603.839645:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b25 from it ffffb10b019a7bf8 00010000:00010000:2.0:1767316603.839648:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.839655:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.839670:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316603.839677:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d135.replay-single, flags=266 00000080:00200000:2.0:1767316603.839684:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 44 00000080:00200008:2.0:1767316603.839692:0:46936:0:(file.c:1001:ll_file_open()) START file d135.replay-single:[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags 704400 00000002:00010000:2.0:1767316603.839708:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x43c:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0104240704401 00010000:00010000:2.0:1767316603.839758:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.839766:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:2.0:1767316603.839776:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1767316603.839781:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d5500 x1853163991587712/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:2.0:1767316603.839790:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316603.839819:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587712:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316603.840020:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.846972:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587712:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316603.847007:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.847018:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.847030:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.847054:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316603.847069:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=200002b, status=0 req@ffff88c5c57d5500 x1853163991587712/t51539607588(51539607588) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:2.0:1767316603.847082:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c57d5500 x1853163991587712/t51539607588(51539607588) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316603.847128:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.847139:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 16 res 8589937617/1084 (0 0) 00000080:00200000:2.0:1767316603.847146:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1767316603.847175:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x43c:0x0], updating i_size 12800 00000080:00010000:2.0:1767316603.847180:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) for lock 0x5295060e33d08b3a 00000080:00010000:2.0:1767316603.847207:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b3a from it ffffb10b019a7bd0 00010000:00010000:2.0:1767316603.847211:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.847221:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316603.847248:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c57d5500 x1853163991587712/t51539607588(51539607588) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:2.0:1767316603.847334:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d135.replay-single:[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1767316603.847342:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908),name=d135.replay-single 00000002:00010000:2.0:1767316603.847365:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x43c:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.847376:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.847388:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.847395:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316603.847400:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00000080:00010000:2.0:1767316603.847403:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) for lock 0x5295060e33d08b2c 00000080:00010000:2.0:1767316603.847407:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b2c from it ffffb10b019a7cd8 00010000:00010000:2.0:1767316603.847409:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.847417:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316603.847464:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) pos/size0/12800 32bit_api 0 00000080:00200000:2.0:1767316603.847536:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) pos/size18446744073709551614/12800 32bit_api 0 00000080:00200000:2.0:1767316603.847562:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:2.0:1767316603.847581:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.3, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:2.0:1767316603.847586:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.3, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:2.0:1767316603.847597:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.3,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.847656:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.847663:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.847673:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316603.847697:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587840:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316603.847791:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.853528:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991587840:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316603.853554:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.853576:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.853585:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.853595:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.853618:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.853639:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.853647:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.853653:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316603.853664:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d4a80 x1853163991587840/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316603.853675:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1767316603.853687:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.853698:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1088 (0 0) 00000080:00200000:2.0:1767316603.853705:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316603.853783:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x440:0x0](ffff88c5c85cdb88): rc = 0 00000080:00200000:2.0:1767316603.853799:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x440:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1767316603.853805:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x440:0x0](ffff88c5c85cdb88) for lock 0x5295060e33d08b41 00000080:00010000:2.0:1767316603.853817:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x440:0x0](ffff88c5c85cdb88) 00000080:00010000:2.0:1767316603.853821:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b41 from it ffffb10b019a7df8 00010000:00010000:2.0:1767316603.853830:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.853839:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.853872:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:2.0:1767316603.853880:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.3, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:2.0:1767316603.853910:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.853927:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.853991:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.854001:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x440:0x0]: losing layout lock 00010000:00010000:2.0:1767316603.854031:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.854052:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.854057:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1767316603.854060:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08b41 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x440:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779969 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.854076:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341d200/0x5295060e33d08b2c lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677992a expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:2.0:1767316603.854269:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1767316603.854278:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d4a80 x1853163991588096/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316603.854303:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588096:192.168.204.145@tcp:36:rm.0 00000100:00100000:2.0:1767316603.854465:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.860512:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588096:192.168.204.145@tcp:36:rm.0 00000080:00200000:2.0:1767316603.860574:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x440:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1767316603.860588:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1767316603.860651:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x440:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1767316603.860659:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x440:0x0](ffff88c5c85cdb88) 00000080:00200000:2.0:1767316603.860739:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:2.0:1767316603.860756:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.9, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:2.0:1767316603.860761:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.9, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:2.0:1767316603.860782:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.9,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.860830:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.860840:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.860849:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316603.860870:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588224:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316603.860936:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.865080:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588224:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316603.865158:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.865180:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.865187:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.865199:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.865210:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.865220:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.865227:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.865232:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316603.865242:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d4380 x1853163991588224/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316603.865252:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1767316603.865285:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.865296:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1094 (0 0) 00000080:00200000:2.0:1767316603.865304:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316603.865348:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x446:0x0](ffff88c5c85d63c8): rc = 0 00000080:00200000:2.0:1767316603.865360:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x446:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1767316603.865365:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x446:0x0](ffff88c5c85d63c8) for lock 0x5295060e33d08b48 00000080:00010000:2.0:1767316603.865376:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x446:0x0](ffff88c5c85d63c8) 00000080:00010000:2.0:1767316603.865379:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b48 from it ffffb10b019a7df8 00010000:00010000:2.0:1767316603.865382:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.865390:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.865419:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:2.0:1767316603.865427:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.9, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:2.0:1767316603.865458:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.865472:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x446:0x0]: losing layout lock 00010000:00010000:2.0:1767316603.865510:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.865518:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1767316603.865520:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606b000/0x5295060e33d08b48 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x446:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67799a1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:2.0:1767316603.865555:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1767316603.865562:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d4380 x1853163991588480/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316603.865584:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588480:192.168.204.145@tcp:36:rm.0 00000100:00100000:2.0:1767316603.865675:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.880198:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588480:192.168.204.145@tcp:36:rm.0 00000080:00200000:3.0:1767316603.887217:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x446:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1767316603.887235:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1767316603.887521:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x446:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1767316603.887529:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x446:0x0](ffff88c5c85d63c8) 00000080:00200000:3.0:1767316603.887619:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:3.0:1767316603.887650:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.10, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:3.0:1767316603.887654:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.10, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:3.0:1767316603.887677:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.10,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:3.0:1767316603.887736:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:3.0:1767316603.887747:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:3.0:1767316603.887756:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:3.0:1767316603.887781:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588608:192.168.204.145@tcp:101:rm.0 00000100:00100000:3.0:1767316603.887943:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426d80 going to sleep for 11 seconds 00000100:00100000:3.0:1767316603.901869:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588608:192.168.204.145@tcp:101:rm.0 00010000:00010000:3.0:1767316603.901896:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.902170:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.902180:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.902190:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.902201:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.902213:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.902221:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.902228:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316603.902323:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c5728700 x1853163991588608/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316603.902335:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1767316603.902347:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.902425:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1095 (0 0) 00000080:00200000:2.0:1767316603.902432:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316603.902478:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x447:0x0](ffff88c5c85db248): rc = 0 00000080:00200000:2.0:1767316603.902492:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x447:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1767316603.902497:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x447:0x0](ffff88c5c85db248) for lock 0x5295060e33d08b4f 00000080:00010000:2.0:1767316603.902506:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x447:0x0](ffff88c5c85db248) 00000080:00010000:2.0:1767316603.902509:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b4f from it ffffb10b019a7df8 00010000:00010000:2.0:1767316603.902512:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.902519:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.902552:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:2.0:1767316603.902559:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.10, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:2.0:1767316603.902591:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.902603:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x447:0x0]: losing layout lock 00010000:00010000:2.0:1767316603.902654:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.902661:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1767316603.902664:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1ba00/0x5295060e33d08b4f lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x447:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a67799d9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:2.0:1767316603.902687:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1767316603.902694:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d5880 x1853163991588864/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316603.902720:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588864:192.168.204.145@tcp:36:rm.0 00000100:00100000:2.0:1767316603.902856:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc000 going to sleep for 11 seconds 00000100:00100000:3.0:1767316603.910971:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588864:192.168.204.145@tcp:36:rm.0 00000080:00200000:3.0:1767316603.911058:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x447:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:3.0:1767316603.911072:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:3.0:1767316603.911576:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x447:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:3.0:1767316603.911585:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x447:0x0](ffff88c5c85db248) 00000080:00200000:3.0:1767316603.911682:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:3.0:1767316603.911701:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.17, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:3.0:1767316603.911706:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.17, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:3.0:1767316603.911730:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.17,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:3.0:1767316603.911789:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:3.0:1767316603.911799:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:3.0:1767316603.911808:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:3.0:1767316603.911833:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588992:192.168.204.145@tcp:101:rm.0 00000100:00100000:3.0:1767316603.911962:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c6426d80 going to sleep for 11 seconds 00000100:00100000:3.0:1767316603.919028:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991588992:192.168.204.145@tcp:101:rm.0 00010000:00010000:3.0:1767316603.919235:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.919749:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.919761:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.919773:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.919784:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.919797:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.919804:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.919830:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316603.919844:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c5728380 x1853163991588992/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316603.919857:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1767316603.919869:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.919879:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1102 (0 0) 00000080:00200000:2.0:1767316603.919886:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316603.919930:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x44e:0x0](ffff88c5c85daa08): rc = 0 00000080:00200000:2.0:1767316603.919943:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x44e:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1767316603.919949:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x44e:0x0](ffff88c5c85daa08) for lock 0x5295060e33d08b56 00000080:00010000:2.0:1767316603.919958:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x44e:0x0](ffff88c5c85daa08) 00000080:00010000:2.0:1767316603.919961:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b56 from it ffffb10b019a7df8 00010000:00010000:2.0:1767316603.919964:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.919972:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.920000:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:2.0:1767316603.920007:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.17, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:2.0:1767316603.920219:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.920243:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44e:0x0]: losing layout lock 00010000:00010000:2.0:1767316603.920428:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.920436:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1767316603.920438:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c4d1b000/0x5295060e33d08b56 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x44e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779a11 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:2.0:1767316603.920464:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1767316603.920471:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d5f80 x1853163991589248/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316603.920494:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991589248:192.168.204.145@tcp:36:rm.0 00000100:00100000:2.0:1767316603.920656:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc240 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.931991:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991589248:192.168.204.145@tcp:36:rm.0 00000080:00200000:2.0:1767316603.932155:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x44e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1767316603.932169:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1767316603.932197:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x44e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1767316603.932204:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x44e:0x0](ffff88c5c85daa08) 00000080:00200000:2.0:1767316603.932513:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:2.0:1767316603.932534:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.4, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:2.0:1767316603.932538:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.4, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:2.0:1767316603.932563:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.4,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316603.932623:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.932643:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316603.932672:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316603.932696:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991589376:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316603.932847:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc240 going to sleep for 11 seconds 00000100:00100000:2.0:1767316603.937806:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991589376:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316603.937838:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.937859:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.937866:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.937876:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.937887:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.937900:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.937908:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316603.937915:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316603.937940:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d7480 x1853163991589376/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316603.937950:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1767316603.937961:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.937972:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1089 (0 0) 00000080:00200000:2.0:1767316603.937979:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316603.938022:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x441:0x0](ffff88c5c85d7448): rc = 0 00000080:00200000:2.0:1767316603.938046:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x441:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:2.0:1767316603.938052:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x441:0x0](ffff88c5c85d7448) for lock 0x5295060e33d08b5d 00000080:00010000:2.0:1767316603.938062:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x441:0x0](ffff88c5c85d7448) 00000080:00010000:2.0:1767316603.938066:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b5d from it ffffb10b019a7df8 00010000:00010000:2.0:1767316603.938069:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.938077:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.938346:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:2.0:1767316603.938462:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.4, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:2.0:1767316603.938497:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316603.938512:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x441:0x0]: losing layout lock 00010000:00010000:2.0:1767316603.938554:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316603.938561:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1767316603.938564:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08b5d lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x441:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779a49 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:2.0:1767316603.938587:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1767316603.938593:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d7480 x1853163991589632/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316603.938615:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991589632:192.168.204.145@tcp:36:rm.0 00000100:00100000:2.0:1767316603.938755:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fc240 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.948441:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991589632:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316603.948508:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x441:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1767316603.948522:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1767316603.948549:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x441:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316603.948557:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x441:0x0](ffff88c5c85d7448) 00000080:00200000:0.0:1767316603.948650:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:0.0:1767316603.948668:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.19, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:0.0:1767316603.948674:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.19, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:0.0:1767316603.948696:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.19,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.948749:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.948759:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.948768:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316603.948790:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991589760:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316603.948889:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.953207:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991589760:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316603.953234:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.953292:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.953300:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.953311:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.953322:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.953335:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.953344:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.953361:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.953373:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6400700 x1853163991589760/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316603.953383:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:0.0:1767316603.953396:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316603.953416:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1104 (0 0) 00000080:00200000:0.0:1767316603.953424:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1767316603.953471:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x450:0x0](ffff88c5c85df448): rc = 0 00000080:00200000:0.0:1767316603.953484:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x450:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1767316603.953489:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x450:0x0](ffff88c5c85df448) for lock 0x5295060e33d08b64 00000080:00010000:0.0:1767316603.953499:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x450:0x0](ffff88c5c85df448) 00000080:00010000:0.0:1767316603.953503:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b64 from it ffffb10b019a7df8 00010000:00010000:0.0:1767316603.953506:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316603.953513:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316603.953543:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:0.0:1767316603.953551:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.19, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:0.0:1767316603.953585:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316603.953599:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x450:0x0]: losing layout lock 00010000:00010000:0.0:1767316603.953641:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316603.953647:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1767316603.953650:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b64 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x450:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779a81 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:0.0:1767316603.953677:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316603.953684:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6400700 x1853163991590016/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316603.953712:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590016:192.168.204.145@tcp:36:rm.0 00000100:00100000:0.0:1767316603.953796:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.961459:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590016:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316603.961514:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x450:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1767316603.961527:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1767316603.961553:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x450:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316603.961560:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x450:0x0](ffff88c5c85df448) 00000080:00200000:0.0:1767316603.961651:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:0.0:1767316603.961667:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.13, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:0.0:1767316603.961672:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.13, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:0.0:1767316603.961693:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.13,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316603.961745:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.961754:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316603.961763:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316603.961782:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590144:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316603.961858:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316603.965924:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590144:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316603.965954:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.965976:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.965998:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.966009:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.966018:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.966042:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.966049:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316603.966056:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316603.966068:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6403480 x1853163991590144/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316603.966078:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:0.0:1767316603.966090:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316603.966119:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1098 (0 0) 00000080:00200000:0.0:1767316603.966126:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1767316603.966175:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x44a:0x0](ffff88c5c85dcb08): rc = 0 00000080:00200000:0.0:1767316603.966187:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x44a:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1767316603.966192:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x44a:0x0](ffff88c5c85dcb08) for lock 0x5295060e33d08b6b 00000080:00010000:0.0:1767316603.966201:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x44a:0x0](ffff88c5c85dcb08) 00000080:00010000:0.0:1767316603.966204:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b6b from it ffffb10b019a7df8 00010000:00010000:0.0:1767316603.966208:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316603.966215:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316603.966248:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316603.966581:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.13, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316603.966615:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316603.966640:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44a:0x0]: losing layout lock 00010000:00010000:1.0:1767316603.966680:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.966698:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316603.966701:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341cc00/0x5295060e33d08b6b lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x44a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779ab9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316603.966731:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316603.966741:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cf800 x1853163991590400/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316603.966764:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590400:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316603.966851:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.972055:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590400:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316603.972131:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x44a:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316603.972144:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316603.972175:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x44a:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316603.972183:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x44a:0x0](ffff88c5c85dcb08) 00000080:00200000:1.0:1767316603.972283:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.972304:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.14, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316603.972309:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.14, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316603.972333:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.14,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.972398:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.972408:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.972417:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.972457:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590528:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.972529:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.977060:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590528:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.977090:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.977135:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.977142:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.977153:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.977163:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.977175:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.977183:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.977190:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.977200:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cca80 x1853163991590528/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.977224:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316603.977237:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.977247:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1099 (0 0) 00000080:00200000:1.0:1767316603.977263:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316603.977309:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x44b:0x0](ffff88c5c85d1988): rc = 0 00000080:00200000:1.0:1767316603.977321:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x44b:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316603.977326:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x44b:0x0](ffff88c5c85d1988) for lock 0x5295060e33d08b72 00000080:00010000:1.0:1767316603.977336:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x44b:0x0](ffff88c5c85d1988) 00000080:00010000:1.0:1767316603.977340:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b72 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316603.977343:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.977363:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316603.977392:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316603.977400:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.14, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316603.977429:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316603.977443:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44b:0x0]: losing layout lock 00010000:00010000:1.0:1767316603.977476:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.977482:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316603.977490:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b72 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x44b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779af1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316603.977516:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316603.977522:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cca80 x1853163991590784/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316603.977545:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590784:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316603.977647:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.983194:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590784:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316603.983265:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x44b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316603.983280:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316603.983308:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x44b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316603.983315:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x44b:0x0](ffff88c5c85d1988) 00000080:00200000:1.0:1767316603.983398:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.983415:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.7, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316603.983420:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.7, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316603.983445:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.7,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.983497:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.983507:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.983515:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.983536:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590912:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.983611:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.991510:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991590912:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316603.991536:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.991556:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.991564:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.991576:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.991587:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.991600:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.991608:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316603.991616:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316603.991640:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cc700 x1853163991590912/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316603.991652:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316603.991687:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.991699:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1092 (0 0) 00000080:00200000:1.0:1767316603.991707:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316603.991752:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x444:0x0](ffff88c5c85d5b88): rc = 0 00000080:00200000:1.0:1767316603.991766:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x444:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316603.991771:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x444:0x0](ffff88c5c85d5b88) for lock 0x5295060e33d08b79 00000080:00010000:1.0:1767316603.991782:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x444:0x0](ffff88c5c85d5b88) 00000080:00010000:1.0:1767316603.991786:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b79 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316603.991790:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.991797:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316603.991826:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316603.991834:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.7, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316603.991865:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316603.991880:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x444:0x0]: losing layout lock 00010000:00010000:1.0:1767316603.991917:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316603.991925:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316603.991928:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b79 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x444:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779b29 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316603.991965:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316603.991972:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cc700 x1853163991591168/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316603.991997:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591168:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316603.992082:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316603.999177:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc700 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591168:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316603.999234:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x444:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316603.999258:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316603.999285:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x444:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316603.999292:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x444:0x0](ffff88c5c85d5b88) 00000080:00200000:1.0:1767316603.999511:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316603.999536:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.5, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316603.999541:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.5, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316603.999562:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.5,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316603.999614:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.999638:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316603.999648:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316603.999669:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591296:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316603.999737:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.005301:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591296:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.005344:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.005381:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.005387:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.005398:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.005409:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.005421:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.005429:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.005435:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.005447:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65ced80 x1853163991591296/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.005459:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.005483:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.005494:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1090 (0 0) 00000080:00200000:1.0:1767316604.005502:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.005546:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x442:0x0](ffff88c5c85caa08): rc = 0 00000080:00200000:1.0:1767316604.005560:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x442:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.005565:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x442:0x0](ffff88c5c85caa08) for lock 0x5295060e33d08b80 00000080:00010000:1.0:1767316604.005575:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x442:0x0](ffff88c5c85caa08) 00000080:00010000:1.0:1767316604.005578:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b80 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.005581:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.005589:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.005618:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.005637:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.5, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.005669:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.005685:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x442:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.005724:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.005731:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.005734:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b80 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x442:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779b61 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.005770:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.005777:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65ced80 x1853163991591552/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.005802:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591552:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.005893:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.014182:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591552:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.014239:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x442:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.014264:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.014291:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x442:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.014298:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x442:0x0](ffff88c5c85caa08) 00000080:00200000:1.0:1767316604.014392:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.014408:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.16, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.014414:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.16, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.014435:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.16,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.014485:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.014494:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.014503:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.014522:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591680:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.014609:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.021159:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591680:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.021197:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.021217:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.021224:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.021234:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.021244:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.021268:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.021276:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.021282:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.021292:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cdf80 x1853163991591680/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.021302:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.021326:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.021336:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1101 (0 0) 00000080:00200000:1.0:1767316604.021344:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.021400:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x44d:0x0](ffff88c5c85d80c8): rc = 0 00000080:00200000:1.0:1767316604.021416:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x44d:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.021421:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x44d:0x0](ffff88c5c85d80c8) for lock 0x5295060e33d08b87 00000080:00010000:1.0:1767316604.021431:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x44d:0x0](ffff88c5c85d80c8) 00000080:00010000:1.0:1767316604.021434:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b87 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.021437:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.021445:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.021475:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.021483:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.16, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.021513:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.021528:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44d:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.021566:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.021573:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.021576:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b87 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x44d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779b99 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.021616:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.021623:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cdf80 x1853163991591936/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.021656:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591936:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.021748:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.026720:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991591936:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.026782:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x44d:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.026796:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.026824:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x44d:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.026831:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x44d:0x0](ffff88c5c85d80c8) 00000080:00200000:1.0:1767316604.026906:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.026922:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.11, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.026927:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.11, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.026951:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.11,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.027002:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.027011:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.027020:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.027041:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592064:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.027141:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.031552:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592064:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.031593:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.031614:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.031622:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.031644:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.031655:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.031668:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.031674:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.031681:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.031693:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cce00 x1853163991592064/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.031704:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.031726:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.031737:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1096 (0 0) 00000080:00200000:1.0:1767316604.031744:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.031788:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x448:0x0](ffff88c5c85ca1c8): rc = 0 00000080:00200000:1.0:1767316604.031800:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x448:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.031805:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x448:0x0](ffff88c5c85ca1c8) for lock 0x5295060e33d08b8e 00000080:00010000:1.0:1767316604.031815:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x448:0x0](ffff88c5c85ca1c8) 00000080:00010000:1.0:1767316604.031818:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b8e from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.031822:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.031830:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.031860:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.031869:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.11, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.031900:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.031915:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x448:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.031950:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.031959:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.031962:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08b8e lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x448:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779bd1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.032000:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.032006:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cce00 x1853163991592320/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.032033:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592320:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.032269:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.040762:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592320:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.040845:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x448:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.040859:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.040889:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x448:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.040897:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x448:0x0](ffff88c5c85ca1c8) 00000080:00200000:1.0:1767316604.040980:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.040999:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.8, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.041004:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.8, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.047176:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.8,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.047325:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.047337:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.047347:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.047374:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592448:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.047472:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.051657:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592448:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.051698:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.051719:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.051727:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.051740:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.051751:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.051762:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.051769:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.051777:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.051788:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cd500 x1853163991592448/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.051799:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.051822:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.051832:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1093 (0 0) 00000080:00200000:1.0:1767316604.051839:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.051878:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x445:0x0](ffff88c5c85de3c8): rc = 0 00000080:00200000:1.0:1767316604.051888:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x445:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.051893:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x445:0x0](ffff88c5c85de3c8) for lock 0x5295060e33d08b95 00000080:00010000:1.0:1767316604.051901:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x445:0x0](ffff88c5c85de3c8) 00000080:00010000:1.0:1767316604.051905:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b95 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.051908:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.051916:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.051943:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.051950:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.8, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.051977:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.051991:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x445:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.052022:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.052029:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.052044:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08b95 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x445:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779c09 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.052080:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.052086:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cd500 x1853163991592704/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.052131:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592704:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.052212:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.057088:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592704:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.057182:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x445:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.057196:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.057225:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x445:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.057233:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x445:0x0](ffff88c5c85de3c8) 00000080:00200000:1.0:1767316604.057330:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.057354:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.2, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.057359:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.2, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.057381:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.2,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.057432:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.057443:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.057452:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.057472:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592832:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.057543:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.061788:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991592832:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.061831:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.061853:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.061861:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.061872:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.061883:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.061895:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.061902:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.061908:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.061919:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cf480 x1853163991592832/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.061930:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.061954:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.061968:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1087 (0 0) 00000080:00200000:1.0:1767316604.061974:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.062019:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x43f:0x0](ffff88c5c85d1148): rc = 0 00000080:00200000:1.0:1767316604.062032:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x43f:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.062051:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43f:0x0](ffff88c5c85d1148) for lock 0x5295060e33d08b9c 00000080:00010000:1.0:1767316604.062061:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x43f:0x0](ffff88c5c85d1148) 00000080:00010000:1.0:1767316604.062064:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b9c from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.062068:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.062075:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.062150:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.062159:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.2, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.062192:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.062206:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x43f:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.062244:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.062263:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.062266:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08b9c lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x43f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779c41 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.062305:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.062312:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cf480 x1853163991593088/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.062337:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593088:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.062435:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.068951:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593088:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.069014:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x43f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.069028:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.069060:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x43f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.069067:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x43f:0x0](ffff88c5c85d1148) 00000080:00200000:1.0:1767316604.069191:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.069208:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.6, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.069213:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.6, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.069235:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.6,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.069301:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.069311:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.069320:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.069342:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593216:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.069414:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.074298:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593216:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.074339:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.074374:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.074383:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.074394:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.074405:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.074416:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.074424:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.074431:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.074443:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cf100 x1853163991593216/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.074453:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.074476:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.074487:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1091 (0 0) 00000080:00200000:1.0:1767316604.074494:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.074537:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x443:0x0](ffff88c5c85ddb88): rc = 0 00000080:00200000:1.0:1767316604.074550:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x443:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.074556:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x443:0x0](ffff88c5c85ddb88) for lock 0x5295060e33d08ba3 00000080:00010000:1.0:1767316604.074565:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x443:0x0](ffff88c5c85ddb88) 00000080:00010000:1.0:1767316604.074568:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08ba3 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.074571:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.074578:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.074608:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.074616:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.6, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.074657:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.074670:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x443:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.074706:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.074712:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.074714:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08ba3 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x443:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779c79 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.074751:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.074758:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cf100 x1853163991593472/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.074781:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593472:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.074857:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.080819:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593472:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.080883:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x443:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.080898:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.080931:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x443:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.080940:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x443:0x0](ffff88c5c85ddb88) 00000080:00200000:1.0:1767316604.081026:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.081045:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.15, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.081051:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.15, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.081075:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.15,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.081194:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.081205:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.081215:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.081236:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593600:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.081308:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.090395:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593600:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.090437:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.090457:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.090464:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.090474:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.090484:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.090496:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.090504:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.090511:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.090522:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cc380 x1853163991593600/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.090533:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.090557:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.090567:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1100 (0 0) 00000080:00200000:1.0:1767316604.090573:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.090614:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x44c:0x0](ffff88c5c85d2a08): rc = 0 00000080:00200000:1.0:1767316604.090638:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x44c:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.090643:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x44c:0x0](ffff88c5c85d2a08) for lock 0x5295060e33d08baa 00000080:00010000:1.0:1767316604.090653:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x44c:0x0](ffff88c5c85d2a08) 00000080:00010000:1.0:1767316604.090656:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08baa from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.090659:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.090667:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.090696:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.090704:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.15, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.090733:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.090746:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44c:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.090778:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.090784:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.090786:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08baa lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x44c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779cb1 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.090823:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.090829:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cc380 x1853163991593856/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.090854:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593856:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.090935:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.099837:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593856:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.099897:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x44c:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.099911:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.099938:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x44c:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.099946:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x44c:0x0](ffff88c5c85d2a08) 00000080:00200000:1.0:1767316604.100027:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.100058:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.20, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.100063:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.20, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.100085:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.20,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.100184:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.100193:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.100203:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.100221:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593984:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.100293:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.107930:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991593984:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.107975:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.107998:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.108006:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.108017:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.108027:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.108038:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.108045:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.108052:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.108063:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65ce300 x1853163991593984/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.108073:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.108133:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.108145:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1105 (0 0) 00000080:00200000:1.0:1767316604.108153:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.108201:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x451:0x0](ffff88c5c85dec08): rc = 0 00000080:00200000:1.0:1767316604.108213:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x451:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.108218:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x451:0x0](ffff88c5c85dec08) for lock 0x5295060e33d08bb1 00000080:00010000:1.0:1767316604.108229:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x451:0x0](ffff88c5c85dec08) 00000080:00010000:1.0:1767316604.108232:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bb1 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.108236:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.108244:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.108284:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.108292:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.20, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.108323:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.108337:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x451:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.108371:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.108378:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.108381:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb1 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x451:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779ce9 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.108421:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.108429:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65ce300 x1853163991594240/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.108454:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991594240:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.108544:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.114555:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991594240:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.114611:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x451:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.114651:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.114677:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x451:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.114683:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x451:0x0](ffff88c5c85dec08) 00000080:00200000:1.0:1767316604.114774:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.114791:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.12, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.114796:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.12, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.114816:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.12,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.114865:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.114874:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.114882:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.114902:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991594368:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.114971:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.119737:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991594368:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.119776:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.119797:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.119805:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.119817:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.119828:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.119839:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.119846:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.119853:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.119865:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cea00 x1853163991594368/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.119878:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.119902:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.119913:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1097 (0 0) 00000080:00200000:1.0:1767316604.119920:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.119964:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x449:0x0](ffff88c5c85dba88): rc = 0 00000080:00200000:1.0:1767316604.119980:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x449:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.119985:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x449:0x0](ffff88c5c85dba88) for lock 0x5295060e33d08bb8 00000080:00010000:1.0:1767316604.119994:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x449:0x0](ffff88c5c85dba88) 00000080:00010000:1.0:1767316604.119998:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bb8 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.120001:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.120008:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.120039:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.120046:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.12, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.120077:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.120090:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x449:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.120165:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.120172:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.120175:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191a800/0x5295060e33d08bb8 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x449:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779d21 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.120217:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.120224:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cea00 x1853163991594624/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.120261:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991594624:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.120349:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.125148:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991594624:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.125208:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x449:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.125221:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.125260:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x449:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.125268:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x449:0x0](ffff88c5c85dba88) 00000080:00200000:1.0:1767316604.125346:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.125481:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.18, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.125487:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.18, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.125508:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.18,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.125560:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.125570:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.125579:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.125599:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991594752:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.125698:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.130349:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991594752:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.130400:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.130421:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.130429:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.130440:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.130451:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.130463:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.130470:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.130477:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.130488:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cd880 x1853163991594752/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.130499:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.130522:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.130533:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1103 (0 0) 00000080:00200000:1.0:1767316604.130540:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.130583:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x44f:0x0](ffff88c5c85d9148): rc = 0 00000080:00200000:1.0:1767316604.130596:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x44f:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.130601:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x44f:0x0](ffff88c5c85d9148) for lock 0x5295060e33d08bbf 00000080:00010000:1.0:1767316604.130610:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x44f:0x0](ffff88c5c85d9148) 00000080:00010000:1.0:1767316604.130613:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bbf from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.130617:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.130634:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.130666:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.130673:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.18, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.130704:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.130718:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x44f:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.130752:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.130758:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.130761:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bbf lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x44f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779d59 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.130795:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.130801:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cd880 x1853163991595008/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.130824:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595008:192.168.204.145@tcp:36:rm.0 00000100:00100000:1.0:1767316604.130909:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.137566:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595008:192.168.204.145@tcp:36:rm.0 00000080:00200000:1.0:1767316604.137644:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x44f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1767316604.137658:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1767316604.137686:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x44f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:1.0:1767316604.137692:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x44f:0x0](ffff88c5c85d9148) 00000080:00200000:1.0:1767316604.137775:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 1 00000080:00200000:1.0:1767316604.137791:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=file.1, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags=0 00000080:00200000:1.0:1767316604.137796:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=file.1, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), intent=getattr 00000002:00010000:1.0:1767316604.137815:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: file.1,[0x0:0x0:0x0]) in obj [0x200000bd1:0x43c:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316604.137865:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.137873:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:1.0:1767316604.137883:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:1.0:1767316604.137909:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595136:192.168.204.145@tcp:101:rm.0 00000100:00100000:1.0:1767316604.137979:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1767316604.143291:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595136:192.168.204.145@tcp:101:rm.0 00010000:00010000:1.0:1767316604.143330:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.143359:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.143367:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.143378:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.143388:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.143399:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.143407:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1767316604.143413:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1767316604.143424:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c65cfb80 x1853163991595136/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/640 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:1.0:1767316604.143435:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:1.0:1767316604.143462:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.143473:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1086 (0 0) 00000080:00200000:1.0:1767316604.143479:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x43c:0x0] suppgids 0 -1: rc 0 00000080:00200000:1.0:1767316604.143522:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x43e:0x0](ffff88c5c85d4b08): rc = 0 00000080:00200000:1.0:1767316604.143535:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x43e:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:1.0:1767316604.143540:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43e:0x0](ffff88c5c85d4b08) for lock 0x5295060e33d08bc6 00000080:00010000:1.0:1767316604.143550:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x43e:0x0](ffff88c5c85d4b08) 00000080:00010000:1.0:1767316604.143553:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bc6 from it ffffb10b019a7df8 00010000:00010000:1.0:1767316604.143557:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.143564:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.143594:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), inode mode 41ed mask 3 00000080:00200000:1.0:1767316604.143601:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=file.1, dir=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00010000:00010000:1.0:1767316604.143641:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:1.0:1767316604.143655:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x43e:0x0]: losing layout lock 00010000:00010000:1.0:1767316604.143694:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:1.0:1767316604.143701:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1767316604.143704:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b400/0x5295060e33d08bc6 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x43e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779d91 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00100000:1.0:1767316604.143741:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:1.0:1767316604.143749:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c65cfb80 x1853163991595392/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 488/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316604.143771:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595392:192.168.204.145@tcp:36:rm.0 00000100:00100000:2.0:1767316604.144666:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca25e9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.156488:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595392:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316604.156576:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x43e:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1767316604.156591:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:0.0:1767316604.156631:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x43e:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316604.156640:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x43e:0x0](ffff88c5c85d4b08) 00000080:00200008:0.0:1767316604.156729:0:46936:0:(file.c:424:ll_file_release()) START file d135.replay-single:[0x200000bd1:0x43c:0x0](ffff88c5c85d0908), flags 704000 00010000:00010000:0.0:1767316604.156753:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.156802:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.156818:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.156827:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.156836:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.156861:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.156867:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.156871:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08b25 00000080:00010000:0.0:1767316604.156878:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b25 from it ffffb10b019a7c10 00010000:00010000:0.0:1767316604.156882:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.156890:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.156904:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316604.156927:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d135.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=0 00000080:00200000:0.0:1767316604.156931:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d135.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:0.0:1767316604.156944:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d135.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.156990:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.156997:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.157006:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316604.157028:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595520:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316604.157744:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.161919:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595520:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316604.161945:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.161976:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.161984:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.161995:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162007:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162018:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162026:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162183:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316604.162195:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6403480 x1853163991595520/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316604.162206:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162215:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1084 (0 0) 00000080:00200000:0.0:1767316604.162221:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1767316604.162318:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x43c:0x0], updating i_size 12800 00000080:00200000:0.0:1767316604.162323:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x43c:0x0](ffff88c5c85d0908): rc = 0 00000080:00010000:0.0:1767316604.162329:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) for lock 0x5295060e33d08bcd 00000080:00010000:0.0:1767316604.162341:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00000080:00010000:0.0:1767316604.162346:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bcd from it ffffb10b019a7e00 00010000:00010000:0.0:1767316604.162349:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162458:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.162489:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.162505:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.162515:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162525:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.162532:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.162535:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.162538:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08b25 00000080:00010000:0.0:1767316604.162542:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08b25 from it ffffb10b019a7d88 00010000:00010000:0.0:1767316604.162545:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162551:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.162559:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1767316604.162571:0:46936:0:(namei.c:2264:ll_rmdir()) VFS Op:name=d135.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:0.0:1767316604.162591:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162614:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.162638:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316604.162680:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c57d5500 x1853163991587712/t51539607588(51539607588) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1767316604.162692:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1767316604.162697:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6403480 x1853163991595776/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1767316604.162719:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595776:192.168.204.145@tcp:35:rm.0 00000100:00100000:0.0:1767316604.162851:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.166318:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991595776:192.168.204.145@tcp:35:rm.0 00000002:00100000:0.0:1767316604.166470:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c57d5500 x1853163991587712/t51539607588(51539607588) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316619 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:0.0:1767316604.166552:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.166562:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 1/0,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.166569:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.166588:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:0.0:1767316604.166592:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08bcd lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779dc9 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.166605:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ae00/0x5295060e33d08b3a lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x43c:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779946 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.166620:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08b25 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a677990e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316604.166649:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316604.166655:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6403480 x1853163991596032/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 512/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316604.166676:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596032:192.168.204.145@tcp:36:rm.0 00000100:00100000:0.0:1767316604.166805:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.171836:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596032:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316604.171894:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x43c:0x0](ffff88c5c85d0908) 00000080:00200000:0.0:1767316604.171966:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.171986:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.172058:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.172069:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.172078:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316604.172138:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596160:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316604.172209:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.177884:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596160:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316604.177911:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.177923:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.177934:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.177944:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.177955:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.177962:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.177969:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316604.177980:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6400a80 x1853163991596160/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316604.177994:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.178023:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1767316604.178030:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316604.178069:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 11776 00000080:00010000:0.0:1767316604.178075:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08bd4 00000080:00010000:0.0:1767316604.178081:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.178085:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bd4 from it ffffb10b019a7b58 00010000:00010000:0.0:1767316604.178088:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.178127:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.178153:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316604.178164:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d202.replay-single, flags=8 00000080:00200008:0.0:1767316604.178179:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1767316604.178186:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88),name=d202.replay-single 00000002:00010000:0.0:1767316604.178209:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.178218:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.178229:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.178235:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.178239:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00010000:0.0:1767316604.178242:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) for lock 0x5295060e33d089dc 00000080:00010000:0.0:1767316604.178256:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089dc from it ffffb10b019a7cb8 00010000:00010000:0.0:1767316604.178266:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.178273:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.178323:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.178330:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.178336:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.178343:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.178349:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.178562:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.178565:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08bd4 00000080:00010000:0.0:1767316604.178570:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bd4 from it ffffb10b019a7bf8 00010000:00010000:0.0:1767316604.178573:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.178580:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.178589:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316604.178594:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d202.replay-single, flags=266 00000080:00200000:0.0:1767316604.178599:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 44 00000080:00200008:0.0:1767316604.178607:0:46936:0:(file.c:1001:ll_file_open()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 704400 00000002:00010000:0.0:1767316604.178631:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0100240704401 00010000:00010000:0.0:1767316604.178687:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.178694:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:0.0:1767316604.178705:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1767316604.178713:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6400a80 x1853163991596416/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1767316604.178722:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316604.178744:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596416:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316604.178923:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.183523:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596416:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316604.183546:0:46936:0:(ldlm_request.c:631:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.183558:0:46936:0:(ldlm_request.c:548:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.183565:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.183571:0:46936:0:(ldlm_lock.c:880:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.183578:0:46936:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.183584:0:46936:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c341c200 already unused, calling callback (ffffffffc1bfab40) 00010000:00010000:0.0:1767316604.183603:0:46936:0:(ldlm_request.c:1595:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.183609:0:46936:0:(ldlm_request.c:1605:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.183616:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.183635:0:46936:0:(ldlm_request.c:1301:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1767316604.183641:0:46936:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.183647:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08bdb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 46936 initiator: MDT0 00000002:00100000:0.0:1767316604.183668:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=2b, status=0 req@ffff88c5c6400a80 x1853163991596416/t51539607611(51539607611) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1767316604.183682:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6400a80 x1853163991596416/t51539607611(51539607611) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1767316604.183694:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316604.183713:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 11776 00000002:00100000:0.0:1767316604.183729:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6400a80 x1853163991596416/t51539607611(51539607611) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 2 fl Complete:RPQU/604/0 rc 301/301 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:0.0:1767316604.183793:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1767316604.183800:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88),name=d202.replay-single 00000002:00010000:0.0:1767316604.183821:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.183835:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.183856:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.183863:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.183868:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00010000:0.0:1767316604.183872:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) for lock 0x5295060e33d089dc 00000080:00010000:0.0:1767316604.183877:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089dc from it ffffb10b019a7cd8 00010000:00010000:0.0:1767316604.183880:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.183887:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.183918:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88) pos/size0/11776 32bit_api 0 00000002:00010000:0.0:1767316604.183930:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000bd1:0x467:0x0], intent: readdir flags 0 00010000:00010000:0.0:1767316604.183936:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.183943:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1767316604.184611:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596544:192.168.204.145@tcp:37:rm.0 00000100:00100000:0.0:1767316604.184778:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.196961:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596544:192.168.204.145@tcp:37:rm.0 00010000:00010000:0.0:1767316604.197959:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.197997:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200008:0.0:1767316604.198224:0:46936:0:(file.c:424:ll_file_release()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 704000 00000002:00100000:0.0:1767316604.198476:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c6400a80 x1853163991596416/t51539607611(51539607611) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 1 fl Complete:RPQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:0.0:1767316604.198489:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1767316604.198495:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6401180 x1853163991596800/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:0.0:1767316604.198521:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596800:192.168.204.145@tcp:35:rm.0 00000100:00100000:0.0:1767316604.198676:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.202309:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991596800:192.168.204.145@tcp:35:rm.0 00000002:00100000:0.0:1767316604.202342:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c6400a80 x1853163991596416/t51539607611(51539607611) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/604/ffffffff rc 301/-1 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1767316604.202418:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.202436:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.202453:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.202466:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.202473:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.202478:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.202482:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08bd4 00000080:00010000:0.0:1767316604.202502:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bd4 from it ffffb10b019a7bf8 00010000:00010000:0.0:1767316604.202506:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.202515:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.202529:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316604.202536:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d202.replay-single, flags=266 00000080:00200000:0.0:1767316604.202543:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 44 00000080:00200008:0.0:1767316604.202550:0:46936:0:(file.c:1001:ll_file_open()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 704400 00000002:00010000:0.0:1767316604.202563:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 0104240704401 00010000:00010000:0.0:1767316604.202606:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.202613:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000020:00100000:0.0:1767316604.202632:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:0.0:1767316604.202638:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6401180 x1853163991597056/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/133032 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:0.0:1767316604.202648:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316604.202667:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597056:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316604.202819:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.210184:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597056:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316604.210216:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.210253:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.210265:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.210276:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 4/1,0 mode: CR/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316604.210292:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=1 disposition=200002b, status=0 req@ffff88c5c6401180 x1853163991597056/t51539607613(51539607613) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1767316604.210304:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6401180 x1853163991597056/t51539607613(51539607613) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316604.210315:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.210325:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 16 res 8589937617/1127 (0 0) 00000080:00200000:0.0:1767316604.210331:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 1 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316604.210351:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 11776 00000080:00010000:0.0:1767316604.210356:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) for lock 0x5295060e33d08be2 00000080:00010000:0.0:1767316604.210363:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08be2 from it ffffb10b019a7bd0 00010000:00010000:0.0:1767316604.210366:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 3/1,0 mode: CR/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.210374:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316604.210408:0:46936:0:(mdc_request.c:810:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff88c5c6401180 x1853163991597056/t51539607613(51539607613) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000080:00200008:0.0:1767316604.210473:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1767316604.210480:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88),name=d202.replay-single 00000002:00010000:0.0:1767316604.210501:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x467:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.210511:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.210522:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.210528:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.210533:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00010000:0.0:1767316604.210537:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) for lock 0x5295060e33d089dc 00000080:00010000:0.0:1767316604.210541:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d089dc from it ffffb10b019a7cd8 00010000:00010000:0.0:1767316604.210544:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.210550:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.210581:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88) pos/size0/11776 32bit_api 0 00000080:00200000:0.0:1767316604.210605:0:46936:0:(dir.c:354:ll_iterate()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88) pos/size18446744073709551614/11776 32bit_api 0 00000080:00200000:0.0:1767316604.210637:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 1 00000080:00200000:0.0:1767316604.210654:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags=0 00000080:00200000:0.0:1767316604.210659:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), intent=getattr 00000002:00010000:0.0:1767316604.210674:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f202.replay-single,[0x0:0x0:0x0]) in obj [0x200000bd1:0x467:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.210714:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.210721:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.210729:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316604.210749:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597184:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316604.210838:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.214810:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597184:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316604.214836:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.214859:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.214866:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.214878:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.214890:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.214901:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.214928:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.214934:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316604.214946:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6400a80 x1853163991597184/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/824 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316604.214956:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 240 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:0.0:1767316604.214968:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.214977:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1129 (0 0) 00000080:00200000:0.0:1767316604.214984:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x467:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1767316604.215040:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x469:0x0](ffff88c5c85d21c8): rc = 0 00000080:00200000:0.0:1767316604.215055:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x469:0x0]: layout version change: 4294967294 -> 2 00000080:00010000:0.0:1767316604.215060:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x469:0x0](ffff88c5c85d21c8) for lock 0x5295060e33d08be9 00000080:00010000:0.0:1767316604.215068:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x469:0x0](ffff88c5c85d21c8) 00000080:00010000:0.0:1767316604.215072:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08be9 from it ffffb10b019a7df8 00010000:00010000:0.0:1767316604.215075:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.215083:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316604.215164:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88), inode mode 41ed mask 3 00000080:00200000:0.0:1767316604.215171:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=f202.replay-single, dir=[0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00010000:00010000:0.0:1767316604.215200:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.215217:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.215302:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316604.215313:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x469:0x0]: losing layout lock 00010000:00010000:0.0:1767316604.215341:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.215348:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.215364:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1767316604.215367:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08be9 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x469:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779e2b expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.215383:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a000/0x5295060e33d089dc lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779541 expref: -99 pid: 45576 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316604.215399:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316604.215406:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6400a80 x1853163991597440/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316604.215442:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597440:192.168.204.145@tcp:36:rm.0 00000100:00100000:0.0:1767316604.215531:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.221238:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597440:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316604.221315:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x469:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:0.0:1767316604.221328:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00020000:00200000:0.0:1767316604.221340:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[1] flags 0x0 00000080:00200000:0.0:1767316604.221371:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x469:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316604.221379:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x469:0x0](ffff88c5c85d21c8) 00000080:00200008:0.0:1767316604.221461:0:46936:0:(file.c:424:ll_file_release()) START file d202.replay-single:[0x200000bd1:0x467:0x0](ffff88c5c997ba88), flags 704000 00010000:00010000:0.0:1767316604.221485:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.221530:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.221550:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.221560:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.221568:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.221575:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.221580:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.221583:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08bd4 00000080:00010000:0.0:1767316604.221590:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bd4 from it ffffb10b019a7c10 00010000:00010000:0.0:1767316604.221593:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.221629:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.221645:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316604.221661:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=0 00000080:00200000:0.0:1767316604.221665:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:0.0:1767316604.221678:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d202.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.221728:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.221736:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.221745:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316604.221764:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597568:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316604.221855:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.225938:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401500 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597568:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316604.225968:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.225986:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.225993:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.226023:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.226045:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.226057:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.226064:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.226071:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316604.226082:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6401500 x1853163991597568/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/584 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316604.226491:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.226505:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1127 (0 0) 00000080:00200000:0.0:1767316604.226512:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316604.226606:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000bd1:0x467:0x0], updating i_size 11776 00000080:00200000:2.0:1767316604.226610:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x467:0x0](ffff88c5c997ba88): rc = 0 00000080:00010000:2.0:1767316604.226617:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) for lock 0x5295060e33d08bf0 00000080:00010000:2.0:1767316604.226636:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00010000:2.0:1767316604.226640:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bf0 from it ffffb10b019a7e00 00010000:00010000:2.0:1767316604.226643:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.226650:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.226682:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316604.226696:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.226704:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.226726:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.226733:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316604.226738:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316604.226741:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08bd4 00000080:00010000:2.0:1767316604.226745:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bd4 from it ffffb10b019a7d88 00010000:00010000:2.0:1767316604.226748:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.226756:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.226764:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:2.0:1767316604.226772:0:46936:0:(namei.c:2264:ll_rmdir()) VFS Op:name=d202.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:2.0:1767316604.226793:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.226813:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.226825:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 2/0,0 mode: CR/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316604.226867:0:46936:0:(mdc_request.c:963:mdc_close()) @@@ matched open req@ffff88c5c6401180 x1853163991597056/t51539607613(51539607613) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000020:00100000:2.0:1767316604.226879:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1767316604.226883:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d5c00 x1853163991597824/t0(0) o35->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:23/10 lens 392/1800 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:2.0:1767316604.226915:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597824:192.168.204.145@tcp:35:rm.0 00000100:00100000:2.0:1767316604.227013:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.233443:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991597824:192.168.204.145@tcp:35:rm.0 00000002:00100000:2.0:1767316604.233478:0:46936:0:(mdc_request.c:831:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff88c5c6401180 x1853163991597056/t51539607613(51539607613) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 1824/608 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00010000:00010000:2.0:1767316604.233552:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.233562:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 1/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.233571:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.233576:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:2.0:1767316604.233578:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ca00/0x5295060e33d08bf0 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x467:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779e63 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.233590:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c200/0x5295060e33d08be2 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x467:0x0].0x0 bits 0x5/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779e08 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.233602:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341c800/0x5295060e33d08bd4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779dec expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:2.0:1767316604.233619:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1767316604.233634:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d5c00 x1853163991598080/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 512/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316604.233675:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598080:192.168.204.145@tcp:36:rm.0 00000100:00100000:2.0:1767316604.233747:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.241055:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598080:192.168.204.145@tcp:36:rm.0 00000080:00200000:2.0:1767316604.241135:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x467:0x0](ffff88c5c997ba88) 00000080:00200000:2.0:1767316604.241210:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316604.241229:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.241297:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.241307:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.241316:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316604.241337:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598208:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316604.241397:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.258852:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598208:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316604.258880:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.258892:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.258902:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.258914:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.258941:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.258948:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.258955:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316604.258965:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d7100 x1853163991598208/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316604.258977:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.258990:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1767316604.258997:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1767316604.259023:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:2.0:1767316604.259041:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08bf7 00000080:00010000:2.0:1767316604.259048:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316604.259052:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bf7 from it ffffb10b019a7b58 00010000:00010000:2.0:1767316604.259055:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.259062:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.259086:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316604.259142:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=f130a.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:2.0:1767316604.259147:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f130a.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:2.0:1767316604.259169:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f130a.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.259217:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.259224:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.259233:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316604.259265:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598336:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316604.259350:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.266314:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598336:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316604.266341:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.266372:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.266380:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.266390:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.266401:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.266433:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.266440:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.266447:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316604.266458:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d7100 x1853163991598336/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/800 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316604.266471:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 216 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1767316604.266481:0:46936:0:(mdc_locks.c:969:mdc_finish_enqueue()) ### DoM lock is returned by: getattr, size: 0 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000002:00010000:2.0:1767316604.266491:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.266501:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1067 (0 0) 00000080:00200000:2.0:1767316604.266508:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316604.266558:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x42b:0x0](ffff88c5c85d42c8): rc = 0 00000080:00200000:2.0:1767316604.266571:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x42b:0x0]: layout version change: 4294967294 -> 2 00000080:00010000:2.0:1767316604.266576:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42b:0x0](ffff88c5c85d42c8) for lock 0x5295060e33d08bfe 00000080:00010000:2.0:1767316604.266588:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42b:0x0](ffff88c5c85d42c8) 00000080:00010000:2.0:1767316604.266592:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bfe from it ffffb10b019a7b20 00010000:00010000:2.0:1767316604.266602:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.266611:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200008:2.0:1767316604.266658:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file f130a.replay-single:[0x200000bd1:0x42b:0x0](ffff88c5c85d42c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1767316604.266665:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x42b:0x0](ffff88c5c85d42c8),name=f130a.replay-single 00000002:00010000:2.0:1767316604.266688:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x42b:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.266699:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.266710:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316604.266716:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316604.266719:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42b:0x0](ffff88c5c85d42c8) 00000080:00010000:2.0:1767316604.266722:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42b:0x0](ffff88c5c85d42c8) for lock 0x5295060e33d08bfe 00000080:00010000:2.0:1767316604.266725:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bfe from it ffffb10b019a7cb8 00010000:00010000:2.0:1767316604.266728:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.266735:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316604.266751:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x42b:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1767316604.266761:0:46936:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200000bd1:0x42b:0x0] 00000002:00010000:2.0:1767316604.266777:0:46936:0:(lustre_osc.h:163:osc_enq2ldlm_flags()) flags: a 00000002:00010000:2.0:1767316604.266782:0:46936:0:(mdc_dev.c:989:mdc_lock_init()) ### lock ffff88c5c4b07030, mdc lock ffff88c5c6d925f0, flags 400001002 00010000:00010000:2.0:1767316604.266802:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.266811:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000002:00010000:2.0:1767316604.266824:0:46936:0:(mdc_dev.c:448:mdc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=0 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00200000:2.0:1767316604.266833:0:46936:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:2.0:1767316604.266837:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 5/1,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.266843:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 4/0,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00200000:2.0:1767316604.266854:0:46936:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1767316604.266863:0:46936:0:(file.c:1686:ll_merge_attr_nolock()) [0x200000bd1:0x42b:0x0] updating i_size 0 i_blocks 3 00000020:00010000:2.0:1767316604.266866:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_release():245 00000020:00010000:2.0:1767316604.266869:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1767316604.266871:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c4b07030 (ffff88c5c2cfdea8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1767316604.266875:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_fini():72 00000020:00010000:2.0:1767316604.266878:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c4b07030 (ffff88c5c2cfd1b8/1) at cl_lock_fini():72 00000080:00200000:2.0:1767316604.266888:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x42b:0x0] ignore/verify layout 0/1, layout version 2 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1767316604.266925:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316604.266933:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.266940:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.266957:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.266962:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316604.266968:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316604.266970:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08bf7 00000080:00010000:2.0:1767316604.266974:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bf7 from it ffffb10b019a7c10 00010000:00010000:2.0:1767316604.266977:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.266983:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.266990:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316604.266999:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f130a.replay-single, flags=0 00000080:00200000:2.0:1767316604.267003:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316604.267009:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.267014:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.267021:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.267027:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316604.267031:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316604.267033:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08bf7 00000080:00010000:2.0:1767316604.267037:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08bf7 from it ffffb10b019a7d68 00010000:00010000:2.0:1767316604.267039:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.267051:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.267059:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:2.0:1767316604.267064:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=f130a.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:2.0:1767316604.267086:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.267139:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x800028400000000 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316604.267156:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x42b:0x0] ignore/verify layout 1/0, layout version 2 restore needed 0 00000080:00200000:2.0:1767316604.267163:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x42b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1767316604.267170:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x42b:0x0]: losing layout lock 00010000:00010000:2.0:1767316604.267203:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x804829400000020 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.267211:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.267216:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1767316604.267219:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08bfe lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x42b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x804829400000020 nid: local remote: 0x60377922a6779ea2 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.267235:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08bf7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779e86 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:2.0:1767316604.267252:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1767316604.267264:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d7100 x1853163991598592/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316604.267287:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598592:192.168.204.145@tcp:36:rm.0 00000100:00100000:2.0:1767316604.267424:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.286557:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7100 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598592:192.168.204.145@tcp:36:rm.0 00000080:00200000:2.0:1767316604.286633:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x42b:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1767316604.286647:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00020000:00200000:2.0:1767316604.286656:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[1] flags 0x0 00000080:00200000:2.0:1767316604.286678:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x42b:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1767316604.286685:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x42b:0x0](ffff88c5c85d42c8) 00000080:00200000:2.0:1767316604.286773:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316604.286790:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.286843:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.286853:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.286862:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316604.286883:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598720:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316604.286949:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.291023:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598720:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316604.291060:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.291072:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.291141:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.291154:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.291167:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.291174:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.291182:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316604.291193:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d6680 x1853163991598720/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316604.291206:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.291220:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1767316604.291227:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1767316604.291265:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:2.0:1767316604.291272:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08c05 00000080:00010000:2.0:1767316604.291278:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316604.291280:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c05 from it ffffb10b019a7b58 00010000:00010000:2.0:1767316604.291283:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.291297:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.291322:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316604.291341:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=f131a.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:2.0:1767316604.291345:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f131a.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:2.0:1767316604.291364:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f131a.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.291402:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.291410:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.291418:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316604.291440:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598848:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316604.291520:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.297142:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991598848:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316604.297170:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.297192:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.297201:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.297228:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.297238:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.297262:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.297269:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.297277:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316604.297289:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d6680 x1853163991598848/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/800 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316604.297299:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 216 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1767316604.297309:0:46936:0:(mdc_locks.c:969:mdc_finish_enqueue()) ### DoM lock is returned by: getattr, size: 8 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000002:00010000:2.0:1767316604.297319:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.297330:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1071 (0 0) 00000080:00200000:2.0:1767316604.297337:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316604.297681:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x42f:0x0](ffff88c5c85d8908): rc = 0 00000080:00200000:2.0:1767316604.297696:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x42f:0x0]: layout version change: 4294967294 -> 2 00000080:00010000:2.0:1767316604.297701:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42f:0x0](ffff88c5c85d8908) for lock 0x5295060e33d08c0c 00000080:00010000:2.0:1767316604.297712:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42f:0x0](ffff88c5c85d8908) 00000080:00010000:2.0:1767316604.297715:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c0c from it ffffb10b019a7b20 00010000:00010000:2.0:1767316604.297719:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.297728:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200008:2.0:1767316604.297764:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file f131a.replay-single:[0x200000bd1:0x42f:0x0](ffff88c5c85d8908), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1767316604.297770:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x42f:0x0](ffff88c5c85d8908),name=f131a.replay-single 00000002:00010000:2.0:1767316604.297790:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x42f:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.297798:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.297807:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316604.297813:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316604.297817:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x42f:0x0](ffff88c5c85d8908) 00000080:00010000:2.0:1767316604.297820:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x42f:0x0](ffff88c5c85d8908) for lock 0x5295060e33d08c0c 00000080:00010000:2.0:1767316604.297823:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c0c from it ffffb10b019a7cb8 00010000:00010000:2.0:1767316604.297826:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.297833:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316604.297855:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x42f:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1767316604.297863:0:46936:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200000bd1:0x42f:0x0] 00000002:00010000:2.0:1767316604.297878:0:46936:0:(lustre_osc.h:163:osc_enq2ldlm_flags()) flags: a 00000002:00010000:2.0:1767316604.297884:0:46936:0:(mdc_dev.c:989:mdc_lock_init()) ### lock ffff88c5c4b07030, mdc lock ffff88c5c6d925f0, flags 400001002 00010000:00010000:2.0:1767316604.297896:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.297905:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000002:00010000:2.0:1767316604.297917:0:46936:0:(mdc_dev.c:448:mdc_lock_lvb_update()) ### acquired size 8, setting rss=8; kms=8, end=0 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00200000:2.0:1767316604.297927:0:46936:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:2.0:1767316604.297930:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 5/1,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.297937:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 4/0,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000020:00200000:2.0:1767316604.297945:0:46936:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:2.0:1767316604.297954:0:46936:0:(file.c:1686:ll_merge_attr_nolock()) [0x200000bd1:0x42f:0x0] updating i_size 8 i_blocks 11 00000020:00010000:2.0:1767316604.297958:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_release():245 00000020:00010000:2.0:1767316604.297961:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_cancel():142 00000020:00010000:2.0:1767316604.297964:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c4b07030 (ffff88c5c2cfdea8/1) at cl_lock_cancel():142 00000020:00010000:2.0:1767316604.297968:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_fini():72 00000020:00010000:2.0:1767316604.297971:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c4b07030 (ffff88c5c2cfd1b8/1) at cl_lock_fini():72 00000080:00200000:2.0:1767316604.297982:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x42f:0x0] ignore/verify layout 0/1, layout version 2 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1767316604.298022:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316604.298348:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.298457:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.298466:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.298473:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316604.298478:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316604.298480:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08c05 00000080:00010000:2.0:1767316604.298485:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c05 from it ffffb10b019a7c10 00010000:00010000:2.0:1767316604.298488:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.298495:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.298505:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316604.298515:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f131a.replay-single, flags=0 00000080:00200000:2.0:1767316604.298519:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316604.298526:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.298532:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.298540:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.298546:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316604.298556:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316604.298559:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08c05 00000080:00010000:2.0:1767316604.298563:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c05 from it ffffb10b019a7d68 00010000:00010000:2.0:1767316604.298566:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.298572:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.298580:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:2.0:1767316604.298586:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=f131a.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:2.0:1767316604.298610:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.298644:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x800028400000000 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316604.298661:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x42f:0x0] ignore/verify layout 1/0, layout version 2 restore needed 0 00000080:00200000:2.0:1767316604.298671:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x42f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1767316604.298679:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x42f:0x0]: losing layout lock 00010000:00010000:2.0:1767316604.298716:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x804829400000020 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.298723:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.298728:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:2.0:1767316604.298731:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c0c lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x42f:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x804829400000020 nid: local remote: 0x60377922a6779ee8 expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.298762:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c05 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779ecc expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:2.0:1767316604.298780:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1767316604.298790:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c57d6680 x1853163991599104/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316604.298814:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991599104:192.168.204.145@tcp:36:rm.0 00000100:00100000:2.0:1767316604.298908:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.308422:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6680 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991599104:192.168.204.145@tcp:36:rm.0 00000080:00200000:2.0:1767316604.308482:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x42f:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1767316604.308495:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00020000:00200000:2.0:1767316604.308504:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[1] flags 0x0 00000080:00200000:2.0:1767316604.308522:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x42f:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:2.0:1767316604.308529:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x42f:0x0](ffff88c5c85d8908) 00000080:00200000:2.0:1767316604.308608:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:2.0:1767316604.308631:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.308683:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.308693:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.308702:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316604.308723:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991599232:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316604.308785:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.315897:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991599232:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316604.315919:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.315928:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.315936:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.315946:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.315958:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.315967:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.315974:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316604.315985:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d6d80 x1853163991599232/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316604.315999:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.316009:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:2.0:1767316604.316014:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:2.0:1767316604.316059:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:2.0:1767316604.316066:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08c13 00000080:00010000:2.0:1767316604.316071:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:2.0:1767316604.316074:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c13 from it ffffb10b019a7b58 00010000:00010000:2.0:1767316604.316078:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.316086:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1767316604.316133:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:2.0:1767316604.316152:0:46936:0:(namei.c:1277:ll_lookup_nd()) VFS Op:name=f132a.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), flags=8 00000080:00200000:2.0:1767316604.316157:0:46936:0:(namei.c:1020:ll_lookup_it()) VFS Op:name=f132a.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8), intent=getattr 00000002:00010000:2.0:1767316604.316176:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f132a.replay-single,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.316219:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.316225:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:2.0:1767316604.316234:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:2.0:1767316604.316264:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991599360:192.168.204.145@tcp:101:rm.0 00000100:00100000:2.0:1767316604.316341:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5ca2fcc00 going to sleep for 11 seconds 00000100:00100000:2.0:1767316604.324602:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991599360:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316604.324659:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.324702:0:46936:0:(ldlm_request.c:717:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.324710:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 4/1,0 mode: --/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.324721:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.324731:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.324743:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.324750:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 5/2,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1767316604.324757:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 4/1,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1767316604.324767:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c57d6d80 x1853163991599360/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 592/848 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:2.0:1767316604.324780:0:46936:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: getattr, lvb_len: 264 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1767316604.324793:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.324814:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589937617/1074 (0 0) 00000080:00200000:2.0:1767316604.324820:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:2.0:1767316604.324867:0:46936:0:(namei.c:151:ll_iget()) got inode: [0x200000bd1:0x432:0x0](ffff88c5c85dc2c8): rc = 0 00000080:00200000:2.0:1767316604.324882:0:46936:0:(file.c:6809:ll_layout_conf()) [0x200000bd1:0x432:0x0]: layout version change: 4294967294 -> 4 00000080:00010000:2.0:1767316604.324887:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x432:0x0](ffff88c5c85dc2c8) for lock 0x5295060e33d08c1a 00000080:00010000:2.0:1767316604.324898:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x432:0x0](ffff88c5c85dc2c8) 00000080:00010000:2.0:1767316604.324902:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c1a from it ffffb10b019a7b20 00010000:00010000:2.0:1767316604.324905:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.324914:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200008:2.0:1767316604.324964:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file f132a.replay-single:[0x200000bd1:0x432:0x0](ffff88c5c85dc2c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:2.0:1767316604.324972:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000bd1:0x432:0x0](ffff88c5c85dc2c8),name=f132a.replay-single 00000002:00010000:2.0:1767316604.324993:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd1:0x432:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1767316604.325001:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.325009:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 2/1,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316604.325015:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:2.0:1767316604.325020:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd1:0x432:0x0](ffff88c5c85dc2c8) 00000080:00010000:2.0:1767316604.325022:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000bd1:0x432:0x0](ffff88c5c85dc2c8) for lock 0x5295060e33d08c1a 00000080:00010000:2.0:1767316604.325026:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c1a from it ffffb10b019a7cb8 00010000:00010000:2.0:1767316604.325036:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 3/1,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1767316604.325043:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1767316604.325059:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x432:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:2.0:1767316604.325069:0:46936:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x200000bd1:0x432:0x0] 00000008:00010000:2.0:1767316604.325089:0:46936:0:(lustre_osc.h:163:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1767316604.325281:0:46936:0:(osc_lock.c:1248:osc_lock_init()) ### lock ffff88c5c640bb30, osc lock ffff88c5c6d925f0, flags 0x400001002 00000008:00010000:2.0:1767316604.325322:0:46936:0:(lustre_osc.h:163:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1767316604.325324:0:46936:0:(osc_lock.c:1248:osc_lock_init()) ### lock ffff88c5c640bb70, osc lock ffff88c5c6d93a20, flags 0x400001002 00000008:00010000:2.0:1767316604.325336:0:46936:0:(lustre_osc.h:163:osc_enq2ldlm_flags()) flags: a 00000008:00010000:2.0:1767316604.325337:0:46936:0:(osc_lock.c:1248:osc_lock_init()) ### lock ffff88c5c640bbb0, osc lock ffff88c5c6d930a0, flags 0x400001002 00010000:00010000:2.0:1767316604.325380:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.325388:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000100:00080000:2.0:1767316604.325687:0:46936:0:(import.c:509:import_select_connection()) lustre-OST0001-osc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1772 00000100:00080000:2.0:1767316604.325700:0:46936:0:(import.c:623:import_select_connection()) lustre-OST0001-osc-ffff88c5c68bd000: import ffff88c5d1a0e800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316604.325710:0:46936:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316604.325723:0:46936:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6a00 x1853163991599616/t0(0) o8->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00010000:00010000:2.0:1767316604.325759:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.325768:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.325803:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.325810:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x400001000 ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000020:00200000:2.0:1767316604.325821:0:46936:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 3 items remaining. 00000100:00100000:1.0:1767316604.326459:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991599616:192.168.204.145@tcp:8:rm.0 00000100:00100000:3.0:1767316604.327174:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991599488:192.168.204.145@tcp:101:rm.0 00000100:00100000:3.0:1767316604.327217:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991599872:192.168.204.145@tcp:101:rm.0 00000100:00100000:3.0:1767316604.327251:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:3.0:1767316604.327260:0:2489:0:(client.c:1829:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff88c5c57d5180 x1853163991599744/t0(0) o101->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 328/400 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1767316604.330807:0:2485:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-ffff88c5c68bd000: committing for last_committed 4294968902 gen 9 00000100:00080000:1.0:1767316604.330824:0:2485:0:(import.c:1123:ptlrpc_connect_interpret()) lustre-OST0001-osc-ffff88c5c68bd000: connect to target with instance 2 00000100:00080000:1.0:1767316604.330833:0:2485:0:(import.c:951:ptlrpc_connect_set_flags()) lustre-OST0001-osc-ffff88c5c68bd000: Resetting ns_connect_flags to server flags: 0xa0425af2e3440038 00000080:00000004:1.0:1767316604.330845:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:1.0:1767316604.330850:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:1.0:1767316604.330856:0:2485:0:(import.c:1204:ptlrpc_connect_interpret()) connected to replayable target: lustre-OST0001_UUID 00000100:00080000:1.0:1767316604.330860:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from CONNECTING to FULL 00000080:00000004:1.0:1767316604.330866:0:2485:0:(lcommon_misc.c:94:cl_ocd_update()) Changing connect_flags: 0xa0425af2e3440038 -> 0xa0425af2e3440038 00000080:00080000:1.0:1767316604.330868:0:2485:0:(lcommon_misc.c:56:cl_init_ea_size()) updating def/max_easize: 72/65536 00000100:00080000:1.0:1767316604.330880:0:2485:0:(recover.c:189:ptlrpc_wake_delayed()) @@@ waking (set 000000001a10b5e4): req@ffff88c5c57d5180 x1853163991599744/t0(0) o101->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 328/400 e 0 to 0 dl 0 ref 1 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'rm.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1767316604.330904:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991599616:192.168.204.145@tcp:8:rm.0 00010000:00010000:2.0:1767316604.336768:0:2489:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 6/2,0 mode: --/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.336793:0:2489:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 6/2,0 mode: --/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.336807:0:2489:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 6/2,0 mode: PR/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.336819:0:2489:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 6/2,0 mode: PR/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000008:00010000:2.0:1767316604.336833:0:2489:0:(osc_lock.c:179:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 5/2,0 mode: PR/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000020:00200000:2.0:1767316604.336843:0:2489:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 2 items remaining. 00010000:00010000:2.0:1767316604.336847:0:2489:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 6/2,0 mode: PR/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.336855:0:2489:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 5/1,0 mode: PR/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.336881:0:2489:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 5/1,0 mode: PR/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.336890:0:2489:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000100:00100000:2.0:1767316604.336909:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991599488:192.168.204.145@tcp:101:rm.0 00010000:00010000:3.0:1767316604.340155:0:2489:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 6/2,0 mode: --/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1767316604.340180:0:2489:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 6/2,0 mode: --/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1767316604.340194:0:2489:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 6/2,0 mode: PR/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1767316604.340206:0:2489:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 6/2,0 mode: PR/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000008:00010000:3.0:1767316604.340220:0:2489:0:(osc_lock.c:179:osc_lock_lvb_update()) ### acquired size 0, setting rss=0; kms=0, end=18446744073709551615 ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 5/2,0 mode: PR/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000020:00200000:3.0:1767316604.340234:0:2489:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:3.0:1767316604.340239:0:2489:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 6/2,0 mode: PR/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1767316604.340259:0:2489:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 5/1,0 mode: PR/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1767316604.340266:0:2489:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 5/1,0 mode: PR/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1767316604.340273:0:2489:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000100:00100000:3.0:1767316604.340308:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991599872:192.168.204.145@tcp:101:rm.0 00010000:00010000:2.0:1767316604.343834:0:2489:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [1048576->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.343856:0:2489:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 6/2,0 mode: --/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.343866:0:2489:0:(ldlm_resource.c:1525:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.343875:0:2489:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000008:00010000:2.0:1767316604.343885:0:2489:0:(osc_lock.c:179:osc_lock_lvb_update()) ### acquired size 2097152, setting rss=2097152; kms=2097152, end=18446744073709551615 ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 5/2,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000020:00200000:2.0:1767316604.343895:0:2489:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00010000:00010000:2.0:1767316604.343916:0:2489:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 6/2,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.343946:0:2489:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.343954:0:2489:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 5/1,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.343961:0:2489:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000100:00100000:2.0:1767316604.343981:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:0:1853163991599744:192.168.204.145@tcp:101:rm.0 00000080:00200000:0.0:1767316604.344082:0:46936:0:(file.c:1686:ll_merge_attr_nolock()) [0x200000bd1:0x432:0x0] updating i_size 2097152 i_blocks 2097 00000020:00010000:0.0:1767316604.344086:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1767316604.344089:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1767316604.344118:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c640bb30 (ffff88c5c2cfdea8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1767316604.344123:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c640bb70 (ffff88c5c2cfd3f8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1767316604.344134:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c640bbb0 (ffff88c5c2cfd878/1) at cl_lock_cancel():142 00000020:00010000:0.0:1767316604.344138:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1767316604.344141:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c640bb30 (ffff88c5c2cfd1b8/1) at cl_lock_fini():72 00000020:00010000:0.0:1767316604.344145:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c640bb70 (ffff88c5c2cfd1b8/1) at cl_lock_fini():72 00000020:00010000:0.0:1767316604.344149:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c640bbb0 (ffff88c5c2cfd1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1767316604.344164:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x432:0x0] ignore/verify layout 0/1, layout version 4 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316604.344208:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.344225:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.344235:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.344244:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.344248:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.344253:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.344256:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08c13 00000080:00010000:0.0:1767316604.344260:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c13 from it ffffb10b019a7c10 00010000:00010000:0.0:1767316604.344262:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.344266:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.344273:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316604.344280:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f132a.replay-single, flags=0 00000080:00200000:0.0:1767316604.344284:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.344288:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.344290:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.344300:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.344303:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.344306:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.344307:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08c13 00000080:00010000:0.0:1767316604.344309:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c13 from it ffffb10b019a7d68 00010000:00010000:0.0:1767316604.344311:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.344315:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.344319:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1767316604.344323:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=f132a.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:0.0:1767316604.344347:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.344367:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 2/0,0 mode: PR/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316604.344376:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x200000bd1:0x432:0x0]: losing layout lock 00010000:00010000:0.0:1767316604.344418:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 1/0,0 mode: --/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.344425:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.344430:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1767316604.344438:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606ac00/0x5295060e33d08c1a lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x432:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779f2e expref: -99 pid: 46936 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.344456:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c606a600/0x5295060e33d08c13 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779f12 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316604.344473:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316604.344481:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6401880 x1853163991600128/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316604.344505:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991600128:192.168.204.145@tcp:36:rm.0 00000100:00100000:0.0:1767316604.344594:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.350518:0:46936:0:(client.c:3081:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff88c5c68bd000: committing for last_committed 51539607616 gen 2 00000100:00100000:0.0:1767316604.350615:0:46936:0:(client.c:3125:ptlrpc_free_committed()) @@@ stopping search req@ffff88c5c57d7100 x1853163991598592/t51539607617(51539607617) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/456 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316604.350638:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401880 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991600128:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316604.350695:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x200000bd1:0x432:0x0] ignore/verify layout 1/0, layout version 4 restore needed 0 00020000:00200000:0.0:1767316604.350709:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00020000:00200000:0.0:1767316604.350719:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[1] flags 0x10 00000080:00200000:0.0:1767316604.350745:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x200000bd1:0x432:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316604.350785:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x200000bd1:0x432:0x0](ffff88c5c85dc2c8) 00000008:00010000:0.0:1767316604.350816:0:46936:0:(osc_object.c:209:osc_object_ast_clear()) ### update lock size 0 blocks 1 [cma]time: 0 0 0 by oinfo size 0 blocks 1 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 1/0,0 mode: PR/PR res: [0x240000400:0x502:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000008:00010000:0.0:1767316604.350838:0:46936:0:(osc_object.c:209:osc_object_ast_clear()) ### update lock size 2097152 blocks 2093 [cma]time: 1767316215 1767316215 0 by oinfo size 2097152 blocks 2093 [cma]time 1767316215 1767316215 0 ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 1/0,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000008:00010000:0.0:1767316604.350864:0:46936:0:(osc_object.c:209:osc_object_ast_clear()) ### update lock size 0 blocks 3 [cma]time: 1767316214 1767316214 0 by oinfo size 0 blocks 3 [cma]time 1767316214 1767316214 0 ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 1/0,0 mode: PR/PR res: [0x240000400:0x503:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00000080:00200000:0.0:1767316604.350929:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.350943:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.350985:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.350991:0:46936:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00010000:00010000:0.0:1767316604.350998:0:46936:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 46936 initiator: MDT0 00000100:00100000:0.0:1767316604.351015:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991600256:192.168.204.145@tcp:101:rm.0 00000100:00100000:0.0:1767316604.352827:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.356960:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991600256:192.168.204.145@tcp:101:rm.0 00010000:00010000:0.0:1767316604.356980:0:46936:0:(ldlm_request.c:702:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.357787:0:46936:0:(ldlm_request.c:730:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.357801:0:46936:0:(ldlm_lock.c:1065:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.357836:0:46936:0:(ldlm_request.c:784:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.357850:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.357857:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.357864:0:46936:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1767316604.357876:0:46936:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff88c5c6400a80 x1853163991600256/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/688 e 0 to 0 dl 1767316620 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1767316604.357889:0:46936:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.357902:0:46936:0:(ldlm_lock.c:1532:ldlm_lock_match_with_skip()) ### not matched ns ffff88c5c55bf400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:0.0:1767316604.357910:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 0 00000080:00200000:0.0:1767316604.357938:0:46936:0:(llite_lib.c:2981:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 10752 00000080:00010000:0.0:1767316604.357945:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08c36 00000080:00010000:0.0:1767316604.357950:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.357953:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c36 from it ffffb10b019a7b58 00010000:00010000:0.0:1767316604.357957:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.357965:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.357997:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316604.358009:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f203.replay-single, flags=8 00000080:00200008:0.0:1767316604.358023:0:46936:0:(file.c:6252:ll_getattr_dentry()) START file f203.replay-single:[0x2000059f1:0x1:0x0](ffff88c5c85d6c08), request_mask 2047, flags 0, foreign 0 00000080:00200000:0.0:1767316604.358029:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x2000059f1:0x1:0x0](ffff88c5c85d6c08),name=f203.replay-single 00000002:00010000:0.0:1767316604.358050:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x2000059f1:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.358059:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 2/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.358069:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 2/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316604.358075:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.358079:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x2000059f1:0x1:0x0](ffff88c5c85d6c08) 00000080:00010000:0.0:1767316604.358082:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x2000059f1:0x1:0x0](ffff88c5c85d6c08) for lock 0x5295060e33d08a4c 00000080:00010000:0.0:1767316604.358086:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08a4c from it ffffb10b019a7cb8 00010000:00010000:0.0:1767316604.358089:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 3/1,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.358159:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 2/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316604.358181:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x2000059f1:0x1:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00000001:00010000:0.0:1767316604.358193:0:46936:0:(glimpse.c:73:cl_glimpse_lock()) Glimpsing inode [0x2000059f1:0x1:0x0] 00000008:00010000:0.0:1767316604.358214:0:46936:0:(lustre_osc.h:163:osc_enq2ldlm_flags()) flags: a 00000008:00010000:0.0:1767316604.358218:0:46936:0:(osc_lock.c:1248:osc_lock_init()) ### lock ffff88c5c4d64eb0, osc lock ffff88c5c2c685f0, flags 0x400001002 00010000:00010000:0.0:1767316604.358234:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 2/1,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316604.358255:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 1048575) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 2/1,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00000020:00200000:0.0:1767316604.358267:0:46936:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 1 items remaining. 00010000:00010000:0.0:1767316604.358271:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 5/1,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x810020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316604.358278:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00000020:00200000:0.0:1767316604.358288:0:46936:0:(cl_io.c:1802:__cl_sync_io_note()) Noting completion of 1 items, 0 items remaining. 00000080:00200000:0.0:1767316604.358298:0:46936:0:(file.c:1686:ll_merge_attr_nolock()) [0x2000059f1:0x1:0x0] updating i_size 0 i_blocks 1 00000020:00010000:0.0:1767316604.358302:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) release lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_release():245 00000020:00010000:0.0:1767316604.358304:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_cancel():142 00000020:00010000:0.0:1767316604.358308:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) cancel lock: ffff88c5c4d64eb0 (ffff88c5c2cfdea8/1) at cl_lock_cancel():142 00000020:00010000:0.0:1767316604.358311:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c5025da8 (ffff88c5c2cfd1b8/0) at cl_lock_fini():72 00000020:00010000:0.0:1767316604.358313:0:46936:0:(cl_lock.c:36:__cl_lock_trace()) destroy lock: ffff88c5c4d64eb0 (ffff88c5c2cfd1b8/1) at cl_lock_fini():72 00000080:00200000:0.0:1767316604.358322:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x2000059f1:0x1:0x0] ignore/verify layout 0/1, layout version 12 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316604.358529:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.358540:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.358547:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.358555:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.358561:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.358571:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.358573:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08c36 00000080:00010000:0.0:1767316604.358578:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c36 from it ffffb10b019a7c10 00010000:00010000:0.0:1767316604.358581:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.358587:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.358595:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 1 00000080:00200000:0.0:1767316604.358604:0:46936:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=f203.replay-single, flags=0 00000080:00200000:0.0:1767316604.358607:0:46936:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:0.0:1767316604.358614:0:46936:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1767316604.358632:0:46936:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.358640:0:46936:0:(ldlm_lock.c:1515:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.358646:0:46936:0:(namei.c:955:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:0.0:1767316604.358649:0:46936:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) 00000080:00010000:0.0:1767316604.358651:0:46936:0:(llite_internal.h:2004:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff88c5c990c2c8) for lock 0x5295060e33d08c36 00000080:00010000:0.0:1767316604.358654:0:46936:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x5295060e33d08c36 from it ffffb10b019a7d68 00010000:00010000:0.0:1767316604.358655:0:46936:0:(ldlm_lock.c:822:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.358660:0:46936:0:(ldlm_lock.c:905:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1767316604.358665:0:46936:0:(file.c:6629:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8), inode mode 41ed mask 3 00000080:00200000:0.0:1767316604.358674:0:46936:0:(namei.c:2354:ll_unlink()) VFS Op:name=f203.replay-single, dir=[0x200000007:0x1:0x0](ffff88c5c990c2c8) 00010000:00010000:0.0:1767316604.358694:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.358712:0:46936:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 2/0,0 mode: PR/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00000080:00200000:0.0:1767316604.358718:0:46936:0:(vvp_object.c:114:vvp_conf_set()) [0x2000059f1:0x1:0x0]: losing layout lock 00010000:00010000:0.0:1767316604.358746:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 1/0,0 mode: --/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.358752:0:46936:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1767316604.358758:0:46936:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 2 locks packed 00010000:00010000:0.0:1767316604.358761:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191b600/0x5295060e33d08a4c lrc: 0/0,0 mode: --/PR res: [0x2000059f1:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a677961a expref: -99 pid: 46197 timeout: 0 lvb_type: 3 00010000:00010000:0.0:1767316604.358781:0:46936:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5c341ce00/0x5295060e33d08c36 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x60377922a6779f82 expref: -99 pid: 46936 timeout: 0 lvb_type: 0 00000020:00100000:0.0:1767316604.358799:0:46936:0:(genops.c:2438:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff88c5c68bd000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1767316604.358807:0:46936:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff88c5c6400a80 x1853163991600512/t0(0) o36->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 504/1800 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316604.358832:0:46936:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991600512:192.168.204.145@tcp:36:rm.0 00000100:00100000:0.0:1767316604.358928:0:46936:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5c4c86000 going to sleep for 11 seconds 00000100:00100000:0.0:1767316604.366510:0:46936:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400a80 pname:cluuid:pid:xid:nid:opc:job rm:5398f0ac-a698-45f7-862d-28236c50e3c9:46936:1853163991600512:192.168.204.145@tcp:36:rm.0 00000080:00200000:0.0:1767316604.366569:0:46936:0:(vvp_io.c:1854:vvp_io_init()) [0x2000059f1:0x1:0x0] ignore/verify layout 1/0, layout version 12 restore needed 0 00020000:00200000:0.0:1767316604.366584:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[0] flags 0x10 00020000:00200000:0.0:1767316604.366609:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[1] flags 0x0 00020000:00200000:0.0:1767316604.366611:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[2] flags 0x0 00020000:00200000:0.0:1767316604.366612:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[3] flags 0x0 00020000:00200000:0.0:1767316604.366614:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[4] flags 0x0 00020000:00200000:0.0:1767316604.366615:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[5] flags 0x0 00020000:00200000:0.0:1767316604.366627:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[6] flags 0x0 00020000:00200000:0.0:1767316604.366629:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[7] flags 0x0 00020000:00200000:0.0:1767316604.366631:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[8] flags 0x0 00020000:00200000:0.0:1767316604.366632:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[9] flags 0x0 00020000:00200000:0.0:1767316604.366634:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[10] flags 0x0 00020000:00200000:0.0:1767316604.366635:0:46936:0:(lov_io.c:860:lov_io_iter_init()) component[11] flags 0x0 00000080:00200000:0.0:1767316604.366656:0:46936:0:(vvp_io.c:289:vvp_io_fini()) [0x2000059f1:0x1:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200000:0.0:1767316604.366663:0:46936:0:(llite_lib.c:1953:ll_clear_inode()) VFS Op:inode=[0x2000059f1:0x1:0x0](ffff88c5c85d6c08) 00000008:00010000:0.0:1767316604.366692:0:46936:0:(osc_object.c:209:osc_object_ast_clear()) ### update lock size 0 blocks 1 [cma]time: 0 0 0 by oinfo size 0 blocks 1 [cma]time 0 0 0 ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 1/0,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x800020000000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316604.681173:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316605.708210:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316606.728908:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316607.754657:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1767316607.848759:0:47204:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: === replay-single: finish cleanup 20:16:46 (1767316606) === 00000100:00100000:0.0:1767316608.647904:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991600768:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316608.648030:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601024:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316608.648062:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1767316608.648069:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991600640:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316608.648131:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991600896:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316608.653846:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991600896:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316608.653869:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991600768:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316608.653880:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601024:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316608.660990:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991600640:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316608.775720:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767316609.519081:0:2480:0:(events.c:340:request_in_callback()) peer: 12345-192.168.204.145@tcp (source: 12345-192.168.204.145@tcp) 00000100:00100000:2.0:1767316609.519190:0:7218:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1853164044880768 00000100:00100000:2.0:1767316609.519201:0:7218:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1853164044880768 00000100:00100000:2.0:1767316609.519265:0:7218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.145@tcp, seq: 1415 00000100:00100000:2.0:1767316609.519275:0:7218:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff88c5c572b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+4:47977:x1853164044880768:12345-192.168.204.145@tcp:104: 00010000:00010000:2.0:1767316609.519293:0:7218:0:(ldlm_lockd.c:2505:ldlm_callback_handler()) ### blocking ast ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1767316609.519374:0:7218:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1767316609.519381:0:7218:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff88c5c572b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb01_000:LOV_OSC_UUID+4:47977:x1853164044880768:12345-192.168.204.145@tcp:104: Request processed in 108us (310us total) trans 0 rc 0/0 00000100:00100000:2.0:1767316609.519388:0:7218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.145@tcp, seq: 1415 00010000:00010000:0.0:1767316609.520392:0:15085:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1767316609.520404:0:15085:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x420000010000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316609.520416:0:15085:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff88c5c606b800 already unused, calling callback (ffffffffc12b30e0) 00010000:00010000:0.0:1767316609.520422:0:15085:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x428400010000 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316609.520483:0:15085:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316609.520490:0:15085:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1767316609.520537:0:15085:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316609.520546:0:15085:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0001-osc-ffff88c5c68bd000 lock: ffff88c5c606b800/0x5295060e33d08c28 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x4c29400010020 nid: local remote: 0x60377922a6779f58 expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1767316609.520568:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316609.520570:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316609.520572:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316609.520581:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767316609.520598:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601152:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1767316609.541047:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601152:192.168.204.145@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1767316609.799304:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767316610.825085:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316611.849755:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316612.872263:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316613.768517:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991601280:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316613.769014:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767316613.769024:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601408:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316613.769342:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601536:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316613.769602:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767316613.769610:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601664:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316613.771073:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601536:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316613.778984:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601664:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316613.896598:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316614.919755:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316615.943628:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316616.969030:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316617.992456:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316618.888348:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991601920:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316618.888463:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991602176:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316618.888500:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1767316618.888506:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2489:1853163991601792:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316618.888529:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991602048:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316618.899384:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991602048:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316618.899403:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991602176:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316619.015882:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316620.042398:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316621.063285:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316622.087394:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316623.111635:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767316624.018535:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991602432:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316624.019387:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991602304:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316624.019500:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991602688:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316624.022455:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991602560:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316624.025632:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991602688:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316624.030250:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991602560:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316624.135479:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316625.160044:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316626.184385:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316627.209130:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316628.232314:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767316629.132617:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:02000000:1.0:1767316629.132627:0:19117:0:(import.c:2080:ptlrpc_disconnect_and_idle_import()) lustre-OST0001-osc-ffff88c5c68bd000: disconnect after 20s idle 00000100:00100000:1.0:1767316629.132735:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991602944:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316629.132886:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1767316629.132894:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991602816:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316629.132926:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991603072:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316629.133244:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991603200:192.168.204.145@tcp:9:kworker.0 00000100:00000400:0.0:1767316629.134424:0:2486:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316613/real 1767316613] req@ffff88c5c65cf100 x1853163991601408/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316629 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316629.134516:0:2486:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c65cf100 x1853163991601408/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316629 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316629.134529:0:2486:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1767316629.134534:0:2486:0:(import.c:181:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff88c5c68bd000: Connection to lustre-MDT0000 (at 192.168.204.145@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:0.0:1767316629.134541:0:2486:0:(import.c:416:ptlrpc_pinger_force()) lustre-MDT0000_UUID: waking up pinger s:DISCONN 00000100:00000400:0.0:1767316629.134618:0:2486:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316613/real 1767316613] req@ffff88c5c65cc380 x1853163991601280/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 1 dl 1767316629 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316629.134667:0:2486:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c65cc380 x1853163991601280/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 1 dl 1767316629 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316629.134675:0:2486:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from FULL to DISCONN 00000100:02020000:0.0:1767316629.134678:0:2486:0:(import.c:185:ptlrpc_set_import_discon()) MGC192.168.204.145@tcp: Connection to MGS (at 192.168.204.145@tcp) was lost; in progress operations using this service will fail 00000100:00080000:0.0:1767316629.134681:0:2486:0:(import.c:191:ptlrpc_set_import_discon()) import MGS@192.168.204.145@tcp for MGC192.168.204.145@tcp not replayable, auto-deactivating 00000100:00080000:0.0:1767316629.134684:0:2486:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID 00000100:00100000:0.0:1767316629.134688:0:2486:0:(client.c:3559:ptlrpc_abort_inflight()) @@@ inflight req@ffff88c5c65cc380 x1853163991601280/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 1 dl 1767316629 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316629.134698:0:2486:0:(client.c:3559:ptlrpc_abort_inflight()) @@@ inflight req@ffff88c5c57d5180 x1853163991601792/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316634 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316629.134728:0:2486:0:(client.c:3559:ptlrpc_abort_inflight()) @@@ inflight req@ffff88c5c6403480 x1853163991602304/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316640 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316629.134737:0:2486:0:(client.c:3559:ptlrpc_abort_inflight()) @@@ inflight req@ffff88c5c65cd500 x1853163991602816/t0(0) o400->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 224/224 e 0 to 0 dl 1767316645 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 10000000:01000000:0.0:1767316629.134757:0:2486:0:(mgc_request.c:1165:mgc_import_event()) import event 0x808001 00000100:00080000:0.0:1767316629.134760:0:2486:0:(pinger.c:192:ptlrpc_pinger_ir_down()) IR down 10000000:01000000:0.0:1767316629.134762:0:2486:0:(mgc_request.c:1165:mgc_import_event()) import event 0x808002 00000100:00080000:0.0:1767316629.134763:0:2486:0:(import.c:416:ptlrpc_pinger_force()) MGS: waking up pinger s:DISCONN 00000100:00100000:1.0:1767316629.134834:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2487:1853163991602816:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316629.135007:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:cf740287-d7f4-4a51-9260-7c6190205bc2:2489:1853163991601792:192.168.204.145@tcp:400:kworker.0 00000100:00080000:2.0:1767316629.135180:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316629.135187:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767316629.135191:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316629.135298:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1758 00000100:00080000:2.0:1767316629.135439:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767316629.135491:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5180 x1853163991603328/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767316629.135612:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991603328:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767316629.135732:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316629.135737:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767316629.135740:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316629.135746:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1772 00000100:00080000:2.0:1767316629.135753:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316629.135764:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316629.135789:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4700 x1853163991603456/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767316629.135914:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991603456:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767316629.136062:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991602304:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316629.136078:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991601408:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316629.136091:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:cf740287-d7f4-4a51-9260-7c6190205bc2:2486:1853163991601280:192.168.204.145@tcp:400:kworker.0 00000100:00080000:3.0:1767316629.138679:0:2485:0:(import.c:1990:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff88c5c65ced80 x1853163991603200/t0(0) o9->lustre-OST0001-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767316639 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767316629.138696:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0e800 lustre-OST0001_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:3.0:1767316629.138701:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991603200:192.168.204.145@tcp:9:kworker.0 00000100:00100000:1.0:1767316629.138813:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991603072:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316629.255445:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316630.285216:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316631.303890:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316632.327280:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316633.351996:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:3.0:1767316634.055163:0:2489:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316618/real 1767316618] req@ffff88c5c57d6d80 x1853163991601920/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316634 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767316634.055193:0:2489:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c57d6d80 x1853163991601920/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316634 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767316634.055202:0:2489:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff88c5c6dc1000 already not connected (conn 39, was 38): CONNECTING 00000100:00100000:3.0:1767316634.055218:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991601920:192.168.204.145@tcp:400:kworker.0 00000100:00080000:0.0:1767316634.247429:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316634.247448:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316634.247451:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316634.247454:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316634.247509:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767316634.247526:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991603584:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316634.253907:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991603584:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316634.375265:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316635.399809:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316636.426340:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316637.447239:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316638.471440:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767316639.367251:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316629/real 1767316629] req@ffff88c5c57d4700 x1853163991603456/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316639 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767316639.367283:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4700 x1853163991603456/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316639 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767316639.367291:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316629/real 1767316629] req@ffff88c5c57d5180 x1853163991603328/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316639 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767316639.367300:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5180 x1853163991603328/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316639 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767316639.367315:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767316639.367337:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767316639.367340:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991603456:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767316639.367344:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767316639.367347:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767316639.367366:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991603328:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767316639.369347:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316639.369360:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767316639.369365:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767316639.369377:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1828 00000100:00080000:1.0:1767316639.369392:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1767316639.369401:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767316639.369450:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cdf80 x1853163991603712/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767316639.369511:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316639.369515:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767316639.369517:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767316639.369521:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1828 00000100:00080000:1.0:1767316639.369525:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 5s 00000100:00080000:1.0:1767316639.369530:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767316639.369540:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767316639.369553:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd500 x1853163991603840/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316639.369634:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1767316639.369655:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991603968:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316639.370496:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991603712:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767316639.370624:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991603840:192.168.204.145@tcp:38:kworker.0 00000100:00100000:1.0:1767316639.372631:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991603968:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316639.495575:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:1.0:1767316640.391325:0:2487:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316624/real 1767316624] req@ffff88c5c6400700 x1853163991602432/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316640 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316640.391377:0:2487:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c6400700 x1853163991602432/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316640 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316640.391387:0:2487:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff88c5c6dc1000 already not connected (conn 40, was 38): CONNECTING 00000100:00100000:1.0:1767316640.391408:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991602432:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316640.519426:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767316641.545071:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316642.570691:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1767316642.816243:0:47708:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: auster : @@@@@@ FAIL: Restart of mds1 failed! 00010000:00010000:1.0:1767316643.591502:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767316644.490174:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316644.490183:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316644.490188:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316644.490191:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316644.490261:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316644.490295:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991604096:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316644.513062:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991604096:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316644.615293:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:0.0:1767316645.511260:0:2487:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316629/real 1767316629] req@ffff88c5c65cce00 x1853163991602944/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316645 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316645.511303:0:2487:0:(client.c:2522:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff88c5c65cce00 x1853163991602944/t0(0) o400->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 224/224 e 0 to 1 dl 1767316645 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316645.511313:0:2487:0:(import.c:213:ptlrpc_set_import_discon()) mdc: import ffff88c5c6dc1000 already not connected (conn 40, was 38): CONNECTING 00000100:00100000:0.0:1767316645.511336:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991602944:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316645.639219:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767316646.665118:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767316647.697655:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:1.0:1767316647.803811:0:48036:0:(file.c:6252:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff88c5c990c2c8), request_mask 2047, flags 0, foreign 0 00000080:00200000:1.0:1767316647.803840:0:48036:0:(file.c:6147:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff88c5c990c2c8),name=/ 00000002:00010000:1.0:1767316647.803867:0:48036:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1767316647.803938:0:48036:0:(ldlm_lock.c:768:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08c3d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 48036 initiator: MDT0 00010000:00010000:1.0:1767316647.803950:0:48036:0:(ldlm_request.c:1022:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08c3d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 48036 initiator: MDT0 00010000:00010000:1.0:1767316647.803960:0:48036:0:(ldlm_request.c:1109:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff88c5c68bd000 lock: ffff88c5d191ba00/0x5295060e33d08c3d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 48036 initiator: MDT0 00000100:00080000:1.0:1767316647.803988:0:48036:0:(client.c:1829:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != CONNECTING) req@ffff88c5c65cc700 x1853163991604224/t0(0) o101->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 576/3504 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1767316647.803999:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316648.711291:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316648.840656:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767316649.607589:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316649.607597:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316649.607600:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316649.607604:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316649.607701:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991604352:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316649.620208:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991604352:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316649.735350:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316649.863691:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316650.762867:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316650.887909:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316651.783898:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316651.911218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316652.811698:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316652.935233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316653.832403:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316653.960081:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767316654.727254:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316639/real 1767316639] req@ffff88c5c65cdf80 x1853163991603712/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316654 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316654.727308:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cdf80 x1853163991603712/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316654 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316654.727319:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316639/real 1767316639] req@ffff88c5c65cd500 x1853163991603840/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316654 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316654.727331:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd500 x1853163991603840/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316654 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316654.727351:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316654.727384:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316654.727389:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991603712:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767316654.727395:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316654.727401:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316654.727404:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991603840:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767316654.727493:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316654.727500:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767316654.727504:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316654.727509:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1838 00000100:00080000:2.0:1767316654.727521:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1767316654.727529:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767316654.727561:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7b80 x1853163991604480/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767316654.727590:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316654.727593:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767316654.727594:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316654.727598:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1838 00000100:00080000:2.0:1767316654.727601:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1767316654.727605:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316654.727615:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316654.727626:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5c00 x1853163991604608/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316654.727724:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316654.727737:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991604736:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316654.728907:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991604480:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767316654.728983:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991604608:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767316654.731397:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991604736:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316654.859353:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316654.983378:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316655.881123:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316656.007243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316656.904169:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316657.031216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316657.929404:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316658.055207:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316658.951449:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316659.079309:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316659.847403:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316659.847412:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316659.847415:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316659.847419:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316659.847508:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1767316659.847531:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991604864:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316659.850506:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991604864:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316659.975483:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316660.103955:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316660.999379:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316661.127262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316662.023283:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316662.152745:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316663.047544:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316663.175161:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316664.072047:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316664.201468:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316664.967228:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316664.967240:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316664.967244:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316664.967269:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316664.967359:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991604992:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316664.973211:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991604992:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316665.095286:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316665.223223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316666.120469:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316666.251958:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316667.143588:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316667.271275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316668.168295:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316668.298592:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316669.191306:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316669.319462:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767316670.095412:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316670.095442:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316670.095445:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316670.095448:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316670.095816:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991605120:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316670.104132:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991605120:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316670.215271:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316670.344306:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316671.241663:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316671.367978:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316672.263433:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316672.391174:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316673.287248:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316673.415227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767316674.183405:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316654/real 1767316654] req@ffff88c5c57d7b80 x1853163991604480/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316674 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316674.183442:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7b80 x1853163991604480/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316674 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316674.183454:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316654/real 1767316654] req@ffff88c5c57d5c00 x1853163991604608/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316674 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316674.183465:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5c00 x1853163991604608/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316674 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316674.183494:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316674.183577:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316674.183582:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991604480:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767316674.183588:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767316674.183722:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316674.183728:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767316674.183730:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316674.183740:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1853 00000100:00080000:2.0:1767316674.183751:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 15s 00000100:00080000:2.0:1767316674.183757:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767316674.183790:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7480 x1853163991605248/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767316674.183804:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316674.183807:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767316674.183809:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316674.183812:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1853 00000100:00080000:2.0:1767316674.183816:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 15s 00000100:00080000:2.0:1767316674.183820:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316674.183829:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316674.183840:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5f80 x1853163991605376/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767316674.183930:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991605504:192.168.204.145@tcp:400:kworker.0 00000100:00080000:0.0:1767316674.184246:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316674.184252:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991604608:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767316674.184271:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991605248:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767316674.184313:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991605376:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767316674.186258:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991605504:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316674.311255:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316674.439221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316675.335262:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316675.464825:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316676.359528:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316676.487305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316677.383965:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316677.511166:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316678.407839:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316678.537628:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316679.303734:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316679.303745:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316679.303748:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316679.303752:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767316679.303871:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991605632:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316679.313058:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991605632:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316679.435888:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316679.559635:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316680.456769:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316680.583790:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316681.479369:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316681.607312:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316682.503220:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316682.631519:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316683.531832:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316683.655235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767316684.424385:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316684.424400:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316684.424403:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316684.424430:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316684.424658:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991605760:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316684.431409:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991605760:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316684.552051:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316684.679343:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316685.575329:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316685.703369:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316686.599302:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316686.727314:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316687.624533:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316687.752889:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316688.647376:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316688.775896:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316689.546470:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316689.546481:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316689.546484:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316689.546488:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316689.546626:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991605888:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316689.566329:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991605888:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316689.672187:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316689.799469:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316690.696176:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316690.830207:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316691.719355:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316691.847286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316692.743367:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316692.871208:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316693.769162:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316693.896814:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316694.663483:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316694.663493:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316694.663510:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316694.663513:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316694.663736:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991606016:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316694.677676:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991606016:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316694.792784:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316694.919343:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316695.819575:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316695.943888:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316696.841014:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316696.967481:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316697.864446:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316697.991762:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316698.887839:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316699.015886:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767316699.783276:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316674/real 1767316674] req@ffff88c5c57d7480 x1853163991605248/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316699 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316699.783329:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7480 x1853163991605248/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316699 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316699.783341:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316674/real 1767316674] req@ffff88c5c57d5f80 x1853163991605376/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316699 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316699.783353:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5f80 x1853163991605376/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316699 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316699.783372:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316699.783395:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316699.783399:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991605248:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767316699.783406:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316699.783412:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316699.783415:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991605376:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767316699.783592:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316699.783598:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767316699.783602:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767316699.783607:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1873 00000100:00080000:1.0:1767316699.783616:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767316699.783622:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767316699.783652:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ce680 x1853163991606144/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767316699.783677:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316699.783680:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767316699.783681:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767316699.783685:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1873 00000100:00080000:1.0:1767316699.783688:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767316699.783692:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767316699.783702:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767316699.783712:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd180 x1853163991606272/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316699.783765:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991606400:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316699.784124:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991606144:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767316699.784170:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991606272:192.168.204.145@tcp:38:kworker.0 00000100:00100000:1.0:1767316699.787312:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991606400:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316699.911684:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316700.041442:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316700.935330:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316701.063626:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316701.959279:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316702.088803:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316702.983253:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316703.113148:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316704.007784:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316704.135273:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316704.905779:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316704.905789:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316704.905792:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316704.905795:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316704.905942:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991606528:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316704.911852:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991606528:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316705.031660:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316705.163641:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316706.055596:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316706.183217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316707.079267:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316707.207222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316708.103679:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316708.233392:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316709.129024:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316709.256932:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316710.023173:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316710.023185:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316710.023188:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316710.023192:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316710.023384:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1767316710.023409:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991606656:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316710.026374:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991606656:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316710.152207:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316710.279383:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316711.175759:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316711.303285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316712.201297:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316712.327264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316713.223692:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316713.351288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316714.247814:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316714.376791:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316715.143582:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316715.143599:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316715.143629:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316715.143634:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767316715.144646:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991606784:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316715.149030:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991606784:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316715.273635:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316715.399379:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316716.306353:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316716.423423:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316717.320152:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316717.447485:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316718.343762:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316718.473602:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316719.367779:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316719.495516:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316720.263617:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316720.263631:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316720.263635:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316720.263638:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316720.263754:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316720.263783:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991606912:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316720.277417:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991606912:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316720.392618:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316720.520499:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316721.415344:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316721.543286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316722.439319:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316722.569509:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316723.463538:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316723.591996:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767316724.360631:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316699/real 1767316699] req@ffff88c5c65ce680 x1853163991606144/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316724 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316724.360672:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ce680 x1853163991606144/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316724 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316724.360683:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316699/real 1767316699] req@ffff88c5c65cd180 x1853163991606272/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316724 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316724.360694:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd180 x1853163991606272/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316724 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316724.360715:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316724.361162:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316724.361168:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991606144:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767316724.361176:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767316724.361236:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316724.361242:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767316724.361246:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316724.361251:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1898 00000100:00080000:3.0:1767316724.361262:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316724.361268:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767316724.361303:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572a680 x1853163991607040/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767316724.361318:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316724.361321:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767316724.361323:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316724.361326:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1898 00000100:00080000:3.0:1767316724.361341:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316724.361345:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767316724.361354:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767316724.361366:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572bb80 x1853163991607168/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316724.361556:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991607296:192.168.204.145@tcp:400:kworker.0 00000100:00080000:0.0:1767316724.362159:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316724.362165:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991606272:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767316724.362194:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991607040:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767316724.362237:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991607168:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767316724.372499:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991607296:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316724.488038:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316724.615309:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316725.511510:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316725.639282:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316726.535582:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316726.663288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316727.561505:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316727.687702:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316728.586032:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316728.711516:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316729.479841:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316729.479859:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316729.479862:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316729.479865:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767316729.480140:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991607424:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316729.485316:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991607424:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316729.607834:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316729.735548:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316730.631391:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316730.759298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316731.655436:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316731.783265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316732.684159:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316732.807285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316733.708659:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316733.831258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316734.601019:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316734.601035:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316734.601037:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316734.601041:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316734.601181:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316734.601212:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991607552:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316734.605168:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991607552:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316734.727474:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316734.855562:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316735.751406:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316735.879420:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316736.775417:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316736.903291:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316737.799881:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316737.927253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316738.823266:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316738.951240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316739.724827:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316739.724837:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316739.724840:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316739.724843:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767316739.724975:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991607680:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316739.749868:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991607680:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316739.847355:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316739.975237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316740.871502:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316740.999318:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316741.896343:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316742.023230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316742.919274:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316743.047236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316743.943385:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316744.071717:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316744.839232:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316744.839244:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316744.839248:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316744.839251:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316744.839332:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316744.839352:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991607808:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316744.844027:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991607808:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316744.967516:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316745.095238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316745.992236:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316746.119793:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316747.015263:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316747.143231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316748.039433:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316748.167474:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316749.065787:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316749.191224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316749.959274:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316749.959284:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316749.959287:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316749.959291:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316749.959461:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316724/real 1767316724] req@ffff88c5c572a680 x1853163991607040/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316749 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316749.959488:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572a680 x1853163991607040/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316749 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316749.959497:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316724/real 1767316724] req@ffff88c5c572bb80 x1853163991607168/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316749 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316749.959505:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572bb80 x1853163991607168/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316749 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316749.959522:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316749.959551:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316749.959570:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991607040:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767316749.959576:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316749.959584:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316749.959587:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991607168:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767316749.959840:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316749.959844:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767316749.959847:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316749.959852:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1923 00000100:00080000:2.0:1767316749.959861:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767316749.959866:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767316749.959889:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7800 x1853163991608064/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767316749.959908:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316749.959927:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767316749.959929:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316749.959933:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1923 00000100:00080000:2.0:1767316749.959938:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767316749.959941:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316749.959949:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316749.959960:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4e00 x1853163991608192/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767316749.960197:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991607936:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316749.960462:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991608064:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767316749.960498:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991608192:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767316749.964500:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991607936:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316750.090036:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316750.215196:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316751.111300:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316751.239287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316752.135440:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316752.263238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316753.159948:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316753.287230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316754.183978:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316754.312890:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316755.080785:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316755.080800:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316755.080807:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316755.080813:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316755.080915:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316755.080934:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991608320:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316755.096292:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991608320:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316755.207368:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316755.336347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316756.232498:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316756.359283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316757.255530:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316757.385115:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316758.280439:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316758.407289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316759.303458:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316759.434162:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316760.199503:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316760.199541:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316760.199545:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316760.199548:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316760.200196:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991608448:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316760.205462:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991608448:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316760.327688:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316760.455268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316761.352910:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316761.482409:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316762.375498:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316762.503944:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316763.399644:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316763.527374:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316764.423487:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316764.551226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316765.319500:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316765.319515:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316765.319518:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316765.319521:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767316765.321153:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991608576:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316765.328008:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991608576:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316765.447694:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316765.575442:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316766.471342:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316766.599237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316767.495284:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316767.623225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316768.523043:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316768.647225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316769.547238:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316769.671242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316770.439390:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316770.439401:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316770.439404:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316770.439407:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316770.439479:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316770.439496:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991608704:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316770.452981:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991608704:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316770.567596:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316770.695528:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316771.591432:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316771.720162:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316772.615441:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316772.743532:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316773.639471:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316773.767760:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767316774.535331:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316749/real 1767316749] req@ffff88c5c57d7800 x1853163991608064/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316774 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316774.535363:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7800 x1853163991608064/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316774 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316774.535373:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316749/real 1767316749] req@ffff88c5c57d4e00 x1853163991608192/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316774 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316774.535410:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4e00 x1853163991608192/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316774 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316774.535428:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316774.535510:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316774.535515:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991608064:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767316774.535521:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316774.535526:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316774.535529:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991608192:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767316774.535647:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316774.535652:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767316774.535655:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316774.535660:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1949 00000100:00080000:3.0:1767316774.535668:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316774.535674:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767316774.535705:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b480 x1853163991608832/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767316774.535741:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316774.535745:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767316774.535746:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316774.535751:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1949 00000100:00080000:3.0:1767316774.535755:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316774.535759:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767316774.535769:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767316774.535781:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728e00 x1853163991608960/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767316774.535832:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316774.535844:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991609088:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316774.541561:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991608832:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767316774.541701:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991608960:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767316774.547230:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991609088:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316774.668118:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316774.792033:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316775.687287:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316775.815241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316776.713466:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316776.839352:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316777.735338:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316777.863222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316778.764427:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316778.887760:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316779.661621:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316779.661631:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316779.661634:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316779.661638:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767316779.661865:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991609216:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316779.669804:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991609216:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767316779.783809:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316779.911283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316780.807341:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316780.935238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316781.832548:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316781.959237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316782.855345:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316782.983242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767316783.879336:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316784.007244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316784.776803:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316784.776814:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316784.776817:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316784.776820:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316784.776922:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316784.776947:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991609344:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316784.782127:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991609344:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316784.910628:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316785.031172:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316785.927500:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316786.055202:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316786.951573:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316787.079296:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316787.978940:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316788.103242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316788.999848:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316789.127816:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316789.897964:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316789.897978:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316789.897982:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316789.897986:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316789.898121:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316789.898151:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991609472:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316789.904850:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991609472:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316790.023614:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316790.152685:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316791.048825:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316791.176031:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316792.074592:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316792.200070:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316793.095723:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316793.224862:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316794.119608:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316794.247203:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767316795.016318:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316795.016329:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316795.016332:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316795.016335:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767316795.016436:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1767316795.016457:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991609600:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316795.026628:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991609600:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316795.143493:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316795.271190:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316796.167354:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316796.295228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316797.193052:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316797.319229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316798.215468:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316798.343295:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767316799.111214:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316774/real 1767316774] req@ffff88c5c572b480 x1853163991608832/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316799 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316799.111254:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b480 x1853163991608832/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316799 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316799.111288:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316774/real 1767316774] req@ffff88c5c5728e00 x1853163991608960/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316799 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316799.111302:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728e00 x1853163991608960/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316799 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316799.111323:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316799.111353:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316799.111357:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991608832:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767316799.111364:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316799.111370:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316799.111372:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991608960:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767316799.111465:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316799.111471:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767316799.111475:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316799.111480:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1973 00000100:00080000:2.0:1767316799.111489:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767316799.111495:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767316799.111532:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5500 x1853163991609728/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767316799.111557:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316799.111560:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767316799.111562:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316799.111574:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1973 00000100:00080000:2.0:1767316799.111578:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767316799.111582:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316799.111592:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316799.111604:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6300 x1853163991609856/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316799.111663:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316799.111676:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991609984:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316799.111793:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991609728:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767316799.111874:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991609856:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767316799.116608:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991609984:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316799.239456:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316799.367229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316800.263574:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316800.391235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316801.287676:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316801.417546:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316802.311461:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316802.439339:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316803.338718:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316803.463689:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316804.231396:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316804.231407:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316804.231410:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316804.231414:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316804.231503:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991610112:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316804.237423:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991610112:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316804.359303:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316804.487210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316805.383936:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316805.512035:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316806.407629:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316806.535258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316807.431913:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316807.559213:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316808.460576:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316808.583223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316809.353272:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316809.353284:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316809.353287:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316809.353290:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316809.353370:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316809.353392:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991610240:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316809.371047:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991610240:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316809.479487:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316809.607253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316810.503811:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316810.631257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316811.527626:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316811.658421:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316812.554599:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316812.682180:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316813.575955:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316813.703507:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767316814.472401:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316814.472411:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316814.472414:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316814.472418:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316814.472605:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991610368:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316814.477819:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991610368:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316814.606565:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316814.727232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316815.630976:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316815.751288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316816.648114:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316816.775428:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316817.678704:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316817.799270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316818.695419:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316818.823226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316819.591547:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316819.591557:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316819.591560:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316819.591563:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316819.591646:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316819.591665:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991610496:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316819.604909:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991610496:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316819.719420:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316819.847268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316820.743810:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316820.874149:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316821.767405:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316821.895238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316822.794849:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316822.919220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316823.815351:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316823.943303:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316824.711217:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316824.711227:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316824.711231:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316824.711235:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316824.711605:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316824.711630:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991610624:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316824.711692:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316799/real 1767316799] req@ffff88c5c57d5500 x1853163991609728/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316824 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316824.711720:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5500 x1853163991609728/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316824 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316824.711731:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316799/real 1767316799] req@ffff88c5c57d6300 x1853163991609856/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316824 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316824.711744:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6300 x1853163991609856/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316824 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316824.711758:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316824.711770:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316824.711774:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991609728:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767316824.711787:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316824.711797:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767316824.711800:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1767316824.714577:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991610624:192.168.204.145@tcp:400:kworker.0 00000100:00080000:1.0:1767316824.715770:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316824.715791:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316824.715796:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991609856:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767316824.719712:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 1998 00000100:00080000:2.0:1767316824.719727:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767316824.719735:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767316824.719784:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7800 x1853163991610752/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767316824.719823:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316824.719829:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767316824.719842:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767316824.719847:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 1998 00000100:00080000:2.0:1767316824.719851:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767316824.719855:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767316824.719864:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767316824.719876:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6a00 x1853163991610880/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767316824.720540:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991610752:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767316824.720643:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991610880:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767316824.846021:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316824.967232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316825.863657:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316825.992794:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316826.893238:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316827.015229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316827.911992:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316828.039249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316828.935482:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316829.063287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316829.831261:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316829.831274:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316829.831278:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316829.831281:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316829.831358:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767316829.831393:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991611008:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316829.834665:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991611008:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316829.960001:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316830.088037:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316830.983718:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316831.111746:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316832.008670:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316832.135245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316833.031461:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316833.159252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316834.059794:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316834.183233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767316834.951400:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316834.951411:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316834.951414:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316834.951428:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316834.951772:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991611136:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316834.961762:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991611136:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316835.079432:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316835.207218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316836.103625:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316836.231701:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316837.127646:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316837.255217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316838.153586:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316838.282210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316839.175333:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316839.303224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316840.072124:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316840.072134:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316840.072155:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316840.072170:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316840.072241:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316840.072272:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991611264:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316840.074719:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991611264:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316840.201162:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316840.327203:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316841.227192:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316841.351467:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316842.249641:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316842.375481:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316843.277964:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316843.401310:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316844.295395:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316844.423664:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316845.191348:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316845.191358:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316845.191361:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316845.191364:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316845.191466:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767316845.191494:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991611392:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316845.203197:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991611392:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316845.319353:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316845.447497:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316846.345252:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316846.471608:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316847.367422:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316847.496685:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316848.391509:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316848.519347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767316849.287281:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316824/real 1767316824] req@ffff88c5c57d7800 x1853163991610752/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316849 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316849.287324:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7800 x1853163991610752/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316849 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316849.287360:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316824/real 1767316824] req@ffff88c5c57d6a00 x1853163991610880/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316849 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316849.287374:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6a00 x1853163991610880/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316849 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316849.287397:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316849.287428:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316849.287433:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991610752:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767316849.287440:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316849.287446:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316849.287450:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991610880:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767316849.287549:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316849.287561:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767316849.287565:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316849.287575:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2023 00000100:00080000:3.0:1767316849.287586:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316849.287594:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767316849.287639:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728380 x1853163991611520/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767316849.287689:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316849.287693:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767316849.287695:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316849.287774:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2023 00000100:00080000:3.0:1767316849.287779:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316849.287783:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767316849.287794:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767316849.287807:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728700 x1853163991611648/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316849.287918:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991611776:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316849.287990:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991611520:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767316849.292533:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991611648:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767316849.299432:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991611776:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316849.415560:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316849.543455:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316850.441443:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316850.567314:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316851.463834:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316851.591688:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316852.488213:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316852.615266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316853.511417:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316853.640546:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316854.408357:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316854.408370:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316854.408374:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316854.408378:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316854.408496:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316854.408520:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991611904:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316854.411152:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991611904:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316854.535831:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316854.663294:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316855.567235:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316855.687227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316856.584167:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316856.711867:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316857.607397:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316857.736437:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316858.631342:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316858.760297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316859.527335:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316859.527345:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316859.527348:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316859.527351:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316859.527421:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316859.527439:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991612032:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316859.530534:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991612032:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316859.655285:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316859.783272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316860.679365:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316860.807238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316861.703431:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316861.831385:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316862.727461:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316862.855221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316863.751297:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316863.879216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316864.647218:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316864.647228:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316864.647231:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316864.647254:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316864.647349:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991612160:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316864.658331:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991612160:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316864.775512:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316864.903639:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316865.799440:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316865.927380:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316866.823368:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316866.951261:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316867.848034:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316867.975244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316868.871449:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316868.999229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316869.767227:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316869.767236:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316869.767239:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316869.767242:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316869.767325:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316869.767350:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991612288:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316869.774929:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991612288:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316869.895362:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316870.023225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316870.921344:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316871.048060:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316871.943492:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316872.071228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316872.967300:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316873.095250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316873.991397:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316874.122208:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316874.888867:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316874.888878:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316874.888882:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316874.888885:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316874.888968:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767316874.888992:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991612416:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316874.890296:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316849/real 1767316849] req@ffff88c5c5728380 x1853163991611520/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316874 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316874.890323:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728380 x1853163991611520/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316874 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316874.890352:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316849/real 1767316849] req@ffff88c5c5728700 x1853163991611648/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316874 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767316874.890366:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728700 x1853163991611648/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316874 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767316874.890393:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316874.890413:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316874.890418:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991611520:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767316874.890424:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316874.890430:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767316874.890433:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991611648:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767316874.890810:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316874.890815:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767316874.890819:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316874.890824:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2048 00000100:00080000:3.0:1767316874.890830:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316874.890836:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767316874.890862:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b100 x1853163991612544/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767316874.890884:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316874.890887:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767316874.890889:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316874.890892:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2048 00000100:00080000:3.0:1767316874.890902:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316874.890906:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767316874.890916:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767316874.890927:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729180 x1853163991612672/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316874.890990:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991612544:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767316874.891066:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991612672:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767316874.904708:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991612416:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316875.016194:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316875.143267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316876.040022:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767316876.167218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316877.064470:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316877.191843:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316878.087409:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316878.215245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316879.111768:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316879.239231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316880.007582:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316880.007592:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316880.007595:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316880.007598:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316880.007698:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767316880.007717:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991612800:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316880.011370:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991612800:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316880.135601:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316880.263231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316881.159288:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316881.287270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316882.183460:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316882.311255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316883.207439:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316883.335292:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316884.231403:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316884.359704:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767316885.127384:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316885.127396:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316885.127399:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316885.127403:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767316885.127488:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1767316885.127517:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991612928:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316885.130436:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991612928:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316885.255533:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316885.383277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316886.279325:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316886.407283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316887.303831:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316887.431227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316888.330283:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316888.455252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316889.351431:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316889.479220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316890.247332:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316890.247342:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316890.247346:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316890.247350:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316890.247425:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316890.247445:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613056:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316890.256478:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613056:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316890.379809:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316890.503850:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316891.399417:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316891.528042:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316892.429415:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316892.551228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316893.447422:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316893.575291:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316894.472124:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316894.606072:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316895.369329:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316895.369341:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316895.369344:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316895.369347:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316895.369771:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613184:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316895.375780:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613184:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316895.495418:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316895.623299:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316896.520373:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316896.649798:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316897.543514:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316897.671239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316898.567494:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316898.695469:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767316899.463791:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316874/real 1767316874] req@ffff88c5c572b100 x1853163991612544/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316899 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316899.463828:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b100 x1853163991612544/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316899 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316899.463839:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316874/real 1767316874] req@ffff88c5c5729180 x1853163991612672/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316899 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316899.463851:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729180 x1853163991612672/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316899 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316899.463870:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316899.470424:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316899.470437:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991612544:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767316899.470450:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316899.470460:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316899.470482:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991612672:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767316899.470570:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316899.470576:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767316899.470579:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316899.470585:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2074 00000100:00080000:3.0:1767316899.470596:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316899.470603:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767316899.470638:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728000 x1853163991613312/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767316899.470664:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316899.470667:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767316899.470669:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316899.470672:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2074 00000100:00080000:3.0:1767316899.470676:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316899.470679:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767316899.470689:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767316899.470700:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729f80 x1853163991613440/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767316899.470775:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991613568:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316899.470922:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991613312:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767316899.470958:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991613440:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767316899.480322:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991613568:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316899.596273:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316899.719247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316900.620995:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316900.743276:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316901.640393:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316901.767693:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316902.663376:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316902.791253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316903.697363:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316903.815257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316904.583341:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316904.583363:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316904.583366:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316904.583389:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316904.583528:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613696:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316904.587607:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613696:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316904.712415:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316904.839249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316905.735321:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316905.864788:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316906.760225:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316906.887220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316907.783443:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316907.911650:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316908.807347:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316908.935305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316909.703671:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316909.703690:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316909.703694:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316909.703698:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316909.703819:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316909.703842:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613824:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316909.714224:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613824:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316909.831480:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316909.961971:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316910.855344:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316910.983757:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316911.882472:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316912.009946:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316912.903598:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316913.031703:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316913.928233:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316914.055335:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316914.823230:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316914.823244:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316914.823248:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316914.823251:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316914.823410:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613952:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316914.836694:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991613952:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316914.951890:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316915.081206:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316915.975325:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316916.103755:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316917.015814:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316917.127923:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316918.026551:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316918.151255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316919.048708:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316919.176820:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316919.945185:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316919.945195:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316919.945198:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316919.945202:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316919.945290:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767316919.945312:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991614080:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316919.959726:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991614080:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316920.071560:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316920.199664:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316921.095574:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316921.223252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316922.121621:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316922.247865:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316923.143484:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316923.271841:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767316924.041169:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316899/real 1767316899] req@ffff88c5c5728000 x1853163991613312/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316924 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316924.041208:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728000 x1853163991613312/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316924 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316924.041221:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316899/real 1767316899] req@ffff88c5c5729f80 x1853163991613440/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316924 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316924.041234:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729f80 x1853163991613440/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316924 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316924.041254:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316924.041286:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316924.041290:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991613312:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767316924.041296:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316924.041302:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316924.041315:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991613440:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767316924.045948:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316924.045959:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767316924.045983:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316924.045994:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2098 00000100:00080000:3.0:1767316924.046008:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316924.046016:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767316924.046054:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572aa00 x1853163991614208/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767316924.046157:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316924.046162:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767316924.046164:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767316924.046169:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2098 00000100:00080000:3.0:1767316924.046173:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767316924.046178:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767316924.046188:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767316924.046201:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729500 x1853163991614336/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767316924.046262:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316924.046280:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991614464:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316924.046862:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991614208:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767316924.046912:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991614336:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767316924.056751:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991614464:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316924.167521:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316924.296066:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316925.191437:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316925.319298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316926.215848:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316926.344008:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316927.239392:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316927.368952:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316928.263346:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316928.391794:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316929.159296:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316929.159309:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316929.159312:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316929.159315:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316929.159409:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316929.159435:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991614592:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316929.163346:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991614592:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316929.287429:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316929.416745:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316930.313008:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316930.439376:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316931.335340:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316931.463269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316932.361687:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316932.487249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316933.385121:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316933.511235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316934.279218:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316934.279238:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316934.279241:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316934.279245:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316934.279374:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991614720:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316934.286329:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991614720:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316934.407621:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316934.535547:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316935.436581:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316935.559226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316936.455459:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316936.583233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316937.486547:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316937.607810:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316938.503462:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316938.631246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316939.401731:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316939.401742:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316939.401745:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316939.401748:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316939.401884:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991614848:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316939.405064:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991614848:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316939.527348:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316939.655681:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316940.551552:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316940.679314:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316941.575305:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316941.703214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316942.600128:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316942.727237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316943.625042:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316943.751228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316944.519899:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316944.519906:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316944.519908:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316944.519911:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316944.519983:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1767316944.519997:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991614976:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316944.525061:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991614976:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316944.647461:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316944.775249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316945.671710:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316945.799228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316946.695352:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316946.823230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316947.723172:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316947.847253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316948.744735:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316948.872756:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767316949.640778:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316924/real 1767316924] req@ffff88c5c572aa00 x1853163991614208/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316949 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316949.640824:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572aa00 x1853163991614208/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316949 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316949.640836:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316924/real 1767316924] req@ffff88c5c5729500 x1853163991614336/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316949 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767316949.640848:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729500 x1853163991614336/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316949 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767316949.640871:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316949.640897:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316949.640902:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991614208:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767316949.640909:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767316949.640915:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767316949.640918:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991614336:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767316949.641511:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316949.641519:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767316949.641522:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767316949.641529:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2123 00000100:00080000:0.0:1767316949.641542:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767316949.641549:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767316949.641589:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401f80 x1853163991615104/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767316949.641616:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316949.641619:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767316949.641621:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767316949.641624:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2123 00000100:00080000:0.0:1767316949.641627:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767316949.641630:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767316949.641656:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767316949.641666:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403480 x1853163991615232/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316949.641717:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991615360:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767316949.642121:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991615104:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767316949.642162:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991615232:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767316949.647552:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991615360:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316949.767640:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316949.899136:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316950.791453:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316950.919287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316951.815403:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316951.943402:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316952.842051:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316952.967232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316953.866031:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316953.991266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316954.759239:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316954.760609:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316954.760613:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316954.760616:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316954.760725:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767316954.760748:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991615488:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316954.776192:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991615488:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316954.888803:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316955.015234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316955.912338:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316956.039378:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316956.935623:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316957.063267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316957.960625:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316958.087319:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316958.983367:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316959.111229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767316959.879213:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316959.879223:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316959.879225:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316959.879229:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316959.879291:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991615616:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316959.884571:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991615616:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767316960.008122:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316960.135371:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316961.031340:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316961.159226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316962.055331:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316962.183223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316963.082035:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316963.208126:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316964.103611:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316964.231233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767316964.999223:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316964.999234:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767316964.999237:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767316964.999240:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767316964.999393:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991615744:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767316965.003719:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991615744:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316965.127426:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316965.255333:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316966.151290:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316966.279954:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316967.175477:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316967.303237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316968.202148:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316968.327269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316969.225380:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316969.357380:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316970.119356:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316970.119372:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316970.119375:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316970.119379:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316970.119485:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1767316970.119511:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991615872:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767316970.122445:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991615872:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316970.247470:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316970.375464:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316971.271399:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316971.399280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316972.298062:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316972.423269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316973.319843:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316973.447320:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767316974.215301:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316949/real 1767316949] req@ffff88c5c6401f80 x1853163991615104/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316974 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316974.215349:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401f80 x1853163991615104/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316974 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316974.215362:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316949/real 1767316949] req@ffff88c5c6403480 x1853163991615232/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316974 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316974.215376:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403480 x1853163991615232/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316974 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767316974.215399:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767316974.215434:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767316974.215438:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991615104:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767316974.215445:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767316974.215451:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767316974.215454:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991615232:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767316974.215572:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316974.215587:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767316974.215592:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767316974.215622:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2148 00000100:00080000:0.0:1767316974.215633:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767316974.215640:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767316974.215677:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400700 x1853163991616000/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767316974.215702:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316974.215706:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767316974.215708:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767316974.215712:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2148 00000100:00080000:0.0:1767316974.215715:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767316974.215719:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767316974.215730:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767316974.215740:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403800 x1853163991616128/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767316974.215823:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767316974.215838:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991616256:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316974.216450:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991616000:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767316974.216507:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991616128:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767316974.222852:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991616256:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316974.344612:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316974.472070:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316975.367483:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316975.495348:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767316976.391572:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316976.519274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316977.415330:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316977.543285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316978.439330:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316978.568928:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316979.336024:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316979.336035:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316979.336038:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316979.336041:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316979.336144:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1767316979.336167:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991616384:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316979.339871:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991616384:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316979.465249:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316979.595823:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316980.487857:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316980.615747:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316981.511609:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316981.639230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316982.535314:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316982.667217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316983.559370:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316983.687476:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316984.455437:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316984.455448:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316984.455451:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316984.455454:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316984.455547:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316984.455571:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991616512:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316984.465463:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991616512:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316984.583545:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316984.711226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316985.607694:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316985.735220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316986.631821:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316986.763186:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316987.655827:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316987.783348:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316988.679308:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316988.807566:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767316989.576150:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316989.576159:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767316989.576180:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767316989.576184:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767316989.576258:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767316989.576277:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991616640:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316989.580391:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991616640:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767316989.703513:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316989.831515:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316990.728134:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316990.855264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316991.753793:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316991.879288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316992.775403:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316992.903218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316993.801247:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316993.927225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767316994.695373:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316994.695384:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767316994.695388:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767316994.695439:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767316994.695508:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767316994.695526:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991616768:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316994.699372:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991616768:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316994.823309:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316994.951431:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316995.847637:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316995.975234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316996.871548:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316997.000058:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767316997.895422:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316998.024132:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767316998.919449:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767316999.049414:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767316999.815264:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316999.815275:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767316999.815277:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316999.815281:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767316999.815656:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991616896:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767316999.817904:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316974/real 1767316974] req@ffff88c5c6400700 x1853163991616000/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316999 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316999.817933:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400700 x1853163991616000/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767316999 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316999.817944:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316974/real 1767316974] req@ffff88c5c6403800 x1853163991616128/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316999 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767316999.817956:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403800 x1853163991616128/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767316999 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767316999.818658:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767316999.818747:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767316999.818753:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991616000:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767316999.818760:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767316999.818859:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316999.818880:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767316999.818882:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767316999.818887:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2173 00000100:00080000:0.0:1767316999.818897:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767316999.818904:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767316999.818934:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401500 x1853163991617024/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767316999.818951:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767316999.818953:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767316999.818955:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767316999.818958:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2173 00000100:00080000:0.0:1767316999.818962:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767316999.818965:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767316999.818975:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767316999.818987:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402680 x1853163991617152/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767316999.819074:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767316999.819079:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991616128:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767316999.819353:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991617024:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767316999.819443:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991617152:192.168.204.145@tcp:38:kworker.0 00000100:00100000:1.0:1767316999.824655:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991616896:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767316999.943478:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317000.071254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317000.969046:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317001.095242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317001.991354:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317002.119190:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317003.015405:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317003.143329:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317004.039425:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317004.167209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317004.935364:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317004.935373:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317004.935376:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317004.935379:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317004.935724:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991617280:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767317004.938781:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991617280:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317005.063344:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317005.191269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317006.089934:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317006.216430:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317007.111675:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317007.239220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317008.136060:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317008.263760:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317009.160198:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317009.287249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317010.055287:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317010.055297:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317010.055301:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317010.055305:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317010.055387:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767317010.055410:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991617408:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317010.065425:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991617408:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317010.184034:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317010.311517:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317011.207402:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317011.335201:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317012.231382:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317012.359215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317013.257577:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317013.385333:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317014.283920:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317014.407225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317015.175387:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317015.175398:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317015.175401:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317015.175405:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317015.175475:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1767317015.175493:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991617536:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317015.181688:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991617536:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317015.303646:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317015.431350:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317016.327265:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317016.455419:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317017.351794:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317017.479337:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317018.375496:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317018.503172:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317019.400069:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317019.527244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317020.297297:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317020.297307:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317020.297309:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317020.297313:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317020.297396:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991617664:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767317020.309040:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991617664:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317020.423378:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317020.551281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317021.447493:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317021.575574:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317022.471759:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317022.599251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317023.495634:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317023.624521:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317024.391541:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316999/real 1767316999] req@ffff88c5c6401500 x1853163991617024/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317024 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317024.391576:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401500 x1853163991617024/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317024 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317024.391588:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767316999/real 1767316999] req@ffff88c5c6402680 x1853163991617152/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317024 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317024.391600:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402680 x1853163991617152/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317024 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317024.391619:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317024.391644:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317024.391666:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991617024:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317024.391672:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317024.391678:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317024.391681:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991617152:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317024.392050:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317024.392057:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317024.392061:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317024.392067:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2199 00000100:00080000:1.0:1767317024.392076:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317024.392084:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317024.392139:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cc000 x1853163991617792/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317024.392165:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317024.392169:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317024.392171:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317024.392174:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2199 00000100:00080000:1.0:1767317024.392178:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317024.392182:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317024.392191:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317024.392210:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf800 x1853163991617920/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317024.392475:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991618048:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767317024.392807:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991617792:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317024.392862:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991617920:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767317024.397413:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991618048:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317024.522382:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317024.647298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317025.543577:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317025.671224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317026.567391:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317026.695217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317027.593272:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317027.719235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317028.615689:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317028.743723:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317029.512050:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317029.512061:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317029.512064:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317029.512066:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317029.512183:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1767317029.512203:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991618176:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317029.515412:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991618176:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317029.639509:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317029.767439:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317030.663746:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317030.791639:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317031.687340:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317031.815254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317032.714934:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317032.839472:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317033.737758:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317033.863247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317034.631405:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317034.631416:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317034.631419:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317034.631422:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767317034.631565:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991618304:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767317034.634726:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991618304:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317034.759569:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317034.887234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317035.783935:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317035.911910:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317036.807451:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317036.935461:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317037.833182:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317037.959294:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317038.855444:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317038.983516:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317039.753523:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317039.753537:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317039.753541:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317039.753545:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317039.753653:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1767317039.753676:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991618432:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317039.757259:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991618432:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317039.879610:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317040.007286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317040.907617:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317041.033051:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317041.928248:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317042.055259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317042.952780:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317043.079237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317043.975579:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317044.103313:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317044.871660:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317044.871676:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317044.871680:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317044.871683:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767317044.871782:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767317044.871814:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991618560:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767317044.876465:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991618560:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317045.000973:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317045.127215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317046.023672:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317046.151294:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317047.050229:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317047.175249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317048.071543:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317048.199316:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317049.095412:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317049.223264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317049.991322:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317024/real 1767317024] req@ffff88c5c65cc000 x1853163991617792/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317049 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317049.991368:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cc000 x1853163991617792/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317049 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317049.991391:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317024/real 1767317024] req@ffff88c5c65cf800 x1853163991617920/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317049 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317049.991404:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1767317049.991406:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf800 x1853163991617920/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317049 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317049.991415:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317049.991421:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317049.991424:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317049.991438:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317049.991473:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317049.991479:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991617792:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317049.991489:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317049.991499:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00080000:0.0:1767317049.991504:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1767317049.991504:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991617920:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767317049.991511:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767317049.991515:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1767317049.991605:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991618688:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317049.997721:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991618688:192.168.204.145@tcp:400:kworker.0 00000100:00080000:0.0:1767317050.001320:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2223 00000100:00080000:0.0:1767317050.001341:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317050.001351:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767317050.001413:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403b80 x1853163991618816/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317050.001478:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317050.001484:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767317050.001488:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317050.001493:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2223 00000100:00080000:0.0:1767317050.001497:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317050.001502:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767317050.001515:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767317050.001530:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400380 x1853163991618944/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317050.001606:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991618816:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317050.001737:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991618944:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317050.125937:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317050.247269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317051.145806:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317051.272116:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317052.170985:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317052.295463:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317053.194251:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317053.319262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317054.216935:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317054.346784:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317055.111438:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317055.111455:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317055.111459:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317055.111462:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317055.111637:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991619072:192.168.204.145@tcp:400:kworker.0 00000100:00100000:2.0:1767317055.114864:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991619072:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317055.239602:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317055.367282:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317056.263343:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317056.391314:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317057.287572:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317057.415291:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317058.311846:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317058.441225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317059.335488:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317059.463235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317060.233283:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317060.233294:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317060.233297:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317060.233301:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317060.233423:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991619200:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317060.245687:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991619200:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317060.361921:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317060.487253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317061.383332:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317061.511319:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317062.407662:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317062.535241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317063.432054:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317063.559215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317064.455867:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317064.586531:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317065.353557:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317065.353570:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317065.353573:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317065.353577:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767317065.353809:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991619328:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317065.356539:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991619328:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317065.481060:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317065.607239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317066.505993:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317066.631363:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317067.532076:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317067.655262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317068.555206:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317068.679220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317069.577688:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317069.704146:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317070.471263:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317070.471274:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317070.471277:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317070.471280:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317070.471365:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767317070.471385:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991619456:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317070.476690:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991619456:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317070.601317:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317070.727212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317071.623560:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317071.752000:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317072.652081:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317072.775266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317073.674125:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317073.799279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317074.695536:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317074.823309:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767317075.591213:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317050/real 1767317050] req@ffff88c5c6403b80 x1853163991618816/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317075 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317075.591272:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403b80 x1853163991618816/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317075 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317075.591284:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317050/real 1767317050] req@ffff88c5c6400380 x1853163991618944/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317075 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317075.591297:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400380 x1853163991618944/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317075 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317075.591316:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317075.591327:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317075.591332:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991618816:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767317075.591338:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317075.591342:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317075.591345:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991618944:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767317075.591430:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317075.591436:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767317075.591439:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317075.591445:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2249 00000100:00080000:0.0:1767317075.591455:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317075.591461:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767317075.591491:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401c00 x1853163991619584/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317075.591528:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317075.591538:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767317075.591540:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317075.591544:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2249 00000100:00080000:0.0:1767317075.591549:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317075.591552:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767317075.591562:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767317075.591574:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402d80 x1853163991619712/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317075.591748:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767317075.591761:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991619840:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317075.591995:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991619584:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767317075.592047:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991619712:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767317075.593815:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991619840:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317075.719401:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317075.847233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317076.743349:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317076.871205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317077.769510:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317077.895223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317078.791372:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317078.920753:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317079.819292:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317079.943251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317080.711441:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317080.711453:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317080.711456:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317080.711460:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317080.711537:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1767317080.711555:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991619968:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317080.718921:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991619968:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317080.848572:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317080.967428:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317081.863660:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317081.991215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317082.889171:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317083.015491:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317083.915073:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317084.039375:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317084.935288:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317085.063224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317085.832561:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317085.832571:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317085.832574:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317085.832578:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767317085.832663:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1767317085.832682:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991620096:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317085.837184:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991620096:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317085.959309:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317086.087344:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317086.983313:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317087.112136:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317088.007332:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317088.135755:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317089.031282:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317089.159309:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317090.055310:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317090.183223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317090.951443:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317090.951457:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317090.951460:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317090.951464:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317090.951549:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767317090.951569:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991620224:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317090.959914:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991620224:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317091.079407:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317091.209148:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317092.104179:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317092.231295:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317093.127389:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317093.255580:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317094.152979:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317094.279185:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317095.175507:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317095.303259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317096.071280:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317096.071290:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317096.071294:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317096.071315:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317096.071383:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1767317096.071404:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991620352:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317096.087402:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991620352:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317096.199448:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317096.329828:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317097.224209:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317097.351241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317098.247310:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317098.376909:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317099.272231:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317099.399241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767317100.167610:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317075/real 1767317075] req@ffff88c5c6401c00 x1853163991619584/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317100 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317100.167647:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401c00 x1853163991619584/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317100 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317100.167658:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317075/real 1767317075] req@ffff88c5c6402d80 x1853163991619712/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317100 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317100.167669:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402d80 x1853163991619712/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317100 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317100.167688:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317100.167716:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317100.167720:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991619584:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767317100.167726:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317100.167731:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317100.167734:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991619712:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767317100.169229:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317100.169238:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767317100.169242:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317100.169250:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2274 00000100:00080000:0.0:1767317100.169260:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317100.169267:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767317100.169304:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402d80 x1853163991620480/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317100.169336:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317100.169340:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767317100.169342:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317100.169346:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2274 00000100:00080000:0.0:1767317100.169349:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317100.169353:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767317100.169362:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767317100.169374:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401c00 x1853163991620608/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317100.169433:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767317100.169447:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991620736:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317100.169686:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991620480:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767317100.172244:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991620608:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767317100.207009:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991620736:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317100.295650:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317100.424614:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317101.320376:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317101.450206:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317102.343658:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317102.472454:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317103.367472:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317103.498582:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317104.395303:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317104.520251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317105.287331:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317105.287343:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317105.287346:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317105.287349:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317105.290870:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991620864:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317105.298381:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991620864:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317105.417521:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317105.543284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317106.439394:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317106.567182:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317107.463978:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317107.592575:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317108.489533:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317108.617502:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317109.511399:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317109.639356:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317110.407354:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317110.407368:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317110.407372:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317110.407376:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317110.407484:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1767317110.407513:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991620992:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317110.419681:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991620992:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317110.536840:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317110.663260:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317111.563188:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317111.687279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317112.583458:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317112.711527:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317113.607445:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317113.735300:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317114.634648:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317114.760593:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317115.527731:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317115.527741:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317115.527760:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317115.527763:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317115.527852:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991621120:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317115.532072:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991621120:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317115.655307:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317115.783238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317116.679337:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317116.811152:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317117.704369:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317117.831388:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317118.727581:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317118.855254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317119.752837:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317119.879377:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317120.647248:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317120.647262:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317120.647265:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317120.647268:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317120.647410:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991621248:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317120.653898:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991621248:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317120.775325:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317120.903336:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317121.799903:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317121.929446:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317122.823740:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317122.951388:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317123.848426:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317123.975229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317124.871603:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317124.999642:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317125.767293:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317125.767305:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317125.767321:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317125.767325:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317125.767431:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991621376:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317125.768970:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317100/real 1767317100] req@ffff88c5c6402d80 x1853163991620480/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317125 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317125.769012:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402d80 x1853163991620480/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317125 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317125.769024:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317100/real 1767317100] req@ffff88c5c6401c00 x1853163991620608/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317125 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317125.769038:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401c00 x1853163991620608/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317125 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317125.769056:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317125.769080:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317125.769084:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991620480:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767317125.769091:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317125.769115:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317125.769119:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991620608:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317125.769795:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317125.769801:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317125.769804:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317125.769811:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2299 00000100:00080000:1.0:1767317125.769819:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317125.769858:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317125.769892:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cca80 x1853163991621504/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317125.769920:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317125.769924:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317125.769926:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317125.769930:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2299 00000100:00080000:1.0:1767317125.769934:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317125.769938:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317125.769948:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317125.769959:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cce00 x1853163991621632/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767317125.770450:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991621504:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767317125.770544:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991621632:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767317125.780372:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991621376:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317125.898424:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317126.023233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317126.920192:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317127.052082:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317127.944187:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317128.072117:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317128.973693:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317129.096169:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317129.993841:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317130.119235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317130.887240:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317130.887252:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317130.887254:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317130.887258:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317130.887336:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991621760:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317130.890471:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991621760:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317131.015563:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317131.143245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317132.039541:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317132.167652:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317133.068123:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317133.191551:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317134.097686:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317134.216088:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317135.112327:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317135.239322:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317136.007243:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317136.007254:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317136.007257:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317136.007262:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317136.007339:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767317136.007369:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991621888:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317136.011835:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991621888:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317136.137237:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317136.263242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317137.171331:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317137.287267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317138.183669:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317138.311671:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317139.212838:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317139.335315:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317140.231771:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317140.359468:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317141.127287:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317141.127298:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317141.127301:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317141.127304:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317141.127470:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991622016:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317141.134693:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991622016:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317141.255542:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317141.384007:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317142.279367:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317142.408747:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317143.303756:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317143.431393:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317144.328810:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317144.455232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317145.351760:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317145.479402:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317146.248525:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317146.248547:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317146.248551:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317146.248555:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317146.248677:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991622144:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317146.254912:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991622144:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317146.375391:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317146.503271:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317147.399313:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317147.527268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317148.426179:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317148.551312:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317149.451713:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317149.575301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317150.343268:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317125/real 1767317125] req@ffff88c5c65cca80 x1853163991621504/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317150 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317150.343399:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cca80 x1853163991621504/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317150 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317150.343411:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317125/real 1767317125] req@ffff88c5c65cce00 x1853163991621632/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317150 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317150.343424:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cce00 x1853163991621632/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317150 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317150.343447:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317150.343650:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317150.343656:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991621504:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317150.343663:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317150.346718:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317150.346733:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767317150.346739:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317150.346751:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2324 00000100:00080000:2.0:1767317150.346766:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317150.346775:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767317150.346827:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4380 x1853163991622272/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767317150.346850:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317150.346854:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767317150.346856:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317150.346860:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2324 00000100:00080000:2.0:1767317150.346864:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317150.346867:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767317150.346877:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767317150.346888:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5880 x1853163991622400/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767317150.347156:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991622528:192.168.204.145@tcp:400:kworker.0 00000100:00080000:0.0:1767317150.350641:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317150.350658:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991621632:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767317150.350713:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991622272:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317150.350844:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991622400:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767317150.351075:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991622528:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317150.471465:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317150.599531:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317151.499254:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317151.623602:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317152.520640:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317152.647284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317153.543457:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317153.671216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317154.567411:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317154.695234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317155.465195:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317155.465205:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317155.465208:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317155.465211:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317155.465283:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767317155.465304:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991622656:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317155.470938:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991622656:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317155.596988:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317155.719212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317156.617534:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317156.743272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317157.639730:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317157.767253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317158.663859:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317158.792623:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317159.700014:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317159.815314:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317160.584199:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317160.584209:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317160.584213:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317160.584216:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317160.584305:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991622784:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317160.594028:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991622784:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317160.711650:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317160.839343:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317161.736060:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317161.866661:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317162.762768:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317162.887973:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317163.783394:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317163.912132:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317164.809969:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317164.935254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317165.703649:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317165.703664:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317165.703668:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317165.703701:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317165.703793:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767317165.703816:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991622912:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317165.709889:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991622912:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317165.831490:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317165.959311:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317166.856872:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317166.985690:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317167.880582:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317168.007246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317168.904080:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317169.032677:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317169.928503:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317170.056503:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317170.823286:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317170.823301:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317170.823304:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317170.823308:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317170.824659:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991623040:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317170.829377:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991623040:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317170.954150:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317171.079544:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317171.978323:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317172.103372:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317173.001090:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317173.128309:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317174.023525:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317174.151384:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317175.048260:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317175.175289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317175.943480:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317150/real 1767317150] req@ffff88c5c57d4380 x1853163991622272/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317175 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317175.943516:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4380 x1853163991622272/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317175 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317175.943538:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317150/real 1767317150] req@ffff88c5c57d5880 x1853163991622400/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317175 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317175.943549:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5880 x1853163991622400/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317175 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317175.943569:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317175.945656:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317175.945685:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767317175.945689:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317175.945998:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2349 00000100:00080000:3.0:1767317175.946012:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317175.946020:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767317175.946059:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572ad80 x1853163991623168/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767317175.946196:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317175.946200:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317175.946630:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1767317175.946648:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991623296:192.168.204.145@tcp:400:kworker.0 00000100:00080000:0.0:1767317175.951648:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317175.951661:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991622272:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317175.951675:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317175.951684:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00080000:1.0:1767317175.952494:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317175.952501:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317175.952505:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317175.952512:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2349 00000100:00100000:0.0:1767317175.952877:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991622400:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317175.953067:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317175.953322:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317175.953337:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767317175.953672:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991623168:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767317175.954660:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf100 x1853163991623424/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317175.956951:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991623296:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317175.957600:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991623424:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317176.072119:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317176.199334:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317177.095835:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317177.224698:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317178.119953:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317178.247296:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317179.145033:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317179.271215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317180.171580:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317180.295197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317181.064955:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317181.064966:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317181.064969:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317181.064973:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317181.065617:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991623552:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317181.094814:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991623552:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317181.195205:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317181.324745:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317182.215332:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317182.343210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317183.241059:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317183.367205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317184.266181:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317184.391731:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317185.287364:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317185.417069:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317186.183510:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317186.183524:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317186.183528:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317186.183532:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317186.183649:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991623680:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317186.186935:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991623680:192.168.204.145@tcp:400:kworker.0 00010000:00010000:2.0:1767317186.313050:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317186.439313:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317187.335830:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317187.463278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317188.361789:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317188.487248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317189.384656:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317189.511901:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317190.407626:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317190.535376:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317191.303274:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317191.303284:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317191.303287:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317191.303290:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767317191.304562:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991623808:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317191.311805:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991623808:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317191.431343:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317191.561922:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317192.455417:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317192.583237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317193.479335:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317193.607228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317194.503335:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317194.631218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317195.528119:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317195.655248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317196.423303:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317196.423315:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317196.423318:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317196.423322:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317196.423394:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1767317196.423412:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991623936:192.168.204.145@tcp:400:kworker.0 00000100:00100000:0.0:1767317196.434164:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991623936:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317196.551479:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317196.679194:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317197.575481:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317197.703229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317198.599339:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317198.728139:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317199.623651:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317199.754573:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317200.519193:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317175/real 1767317175] req@ffff88c5c65cf100 x1853163991623424/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317200 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317200.519235:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf100 x1853163991623424/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317200 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317200.519246:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317175/real 1767317175] req@ffff88c5c572ad80 x1853163991623168/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317200 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317200.519259:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572ad80 x1853163991623168/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317200 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317200.519279:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317200.519364:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317200.519369:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991623424:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767317200.519376:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317200.519401:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317200.519406:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767317200.519409:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317200.519416:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2375 00000100:00080000:3.0:1767317200.519425:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317200.519432:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767317200.519462:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728000 x1853163991624064/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767317200.519475:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317200.519478:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767317200.519479:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317200.519482:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2375 00000100:00080000:3.0:1767317200.519486:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317200.519490:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767317200.519500:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767317200.519511:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729180 x1853163991624192/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767317200.519565:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991624320:192.168.204.145@tcp:400:kworker.0 00000100:00080000:2.0:1767317200.521335:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317200.521348:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991623168:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317200.521391:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991624064:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317200.521464:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991624192:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767317200.530706:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991624320:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317200.647401:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317200.775438:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317201.671311:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317201.800309:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317202.695313:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317202.823234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317203.719476:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317203.848318:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317204.743824:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317204.871333:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317205.639431:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317205.639443:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317205.639446:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317205.639469:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317205.639565:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1767317205.639584:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991624448:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317205.645211:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991624448:192.168.204.145@tcp:400:kworker.0 00010000:00010000:0.0:1767317205.768253:0:48104:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 3/3 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1767317205.774268:0:7228:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 3 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1767317205.774285:0:7228:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767317205.774317:0:7228:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767317205.774326:0:7228:0:(ldlm_request.c:1285:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x28400000000 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767317205.774370:0:7228:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767317205.774580:0:7228:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767317205.774588:0:7228:0:(ldlm_request.c:1332:__ldlm_pack_lock()) ### packing ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767317205.774593:0:7228:0:(ldlm_request.c:1378:ldlm_cancel_pack()) 3 locks packed 00010000:00010000:1.0:1767317205.774626:0:7228:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5d191a000/0x5295060e33d08a6f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x582:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779621 expref: -99 pid: 46197 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767317205.774668:0:7228:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606b400/0x5295060e33d08c2f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x503:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779f4a expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767317205.774679:0:7228:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff88c5c68bd000 lock: ffff88c5c606a400/0x5295060e33d08c21 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x502:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x4829400000020 nid: local remote: 0x60377922a6779f3c expref: -99 pid: 46936 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1767317205.774691:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767317205.774693:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767317205.774695:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317205.774709:0:2487:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1767317205.774729:0:2487:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991624576:192.168.204.145@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1767317205.779502:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317205.798378:0:2487:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2487:1853163991624576:192.168.204.145@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1767317205.895227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317206.792347:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317206.919331:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317207.818787:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317207.943274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317208.839324:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317208.967480:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767317209.863333:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317209.991228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317210.760416:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317210.760426:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317210.760429:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317210.760441:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317210.760633:0:2489:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1767317210.760653:0:2489:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991624704:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317210.767648:0:2489:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5398f0ac-a698-45f7-862d-28236c50e3c9:2489:1853163991624704:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317210.887391:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317211.015499:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317211.911290:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317212.039306:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317212.938747:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317213.063318:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317213.959392:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317214.087416:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317214.983630:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317215.111230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317215.879519:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317215.879530:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317215.879533:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317215.879537:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767317215.879613:0:2486:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1767317215.879638:0:2486:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991624832:192.168.204.145@tcp:400:kworker.0 00000100:00100000:1.0:1767317215.888987:0:2486:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2486:1853163991624832:192.168.204.145@tcp:400:kworker.0 00010000:00010000:3.0:1767317216.007520:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317216.136690:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317217.034521:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317217.160715:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317218.055292:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317218.183212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317219.079311:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317219.207231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317220.110232:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317220.231223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317220.999706:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317220.999717:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317220.999720:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317220.999733:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317220.999810:0:2488:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1767317220.999830:0:2488:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991624960:192.168.204.145@tcp:400:kworker.0 00000100:00100000:3.0:1767317221.003292:0:2488:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5398f0ac-a698-45f7-862d-28236c50e3c9:2488:1853163991624960:192.168.204.145@tcp:400:kworker.0 00010000:00010000:1.0:1767317221.128188:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317221.256645:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317222.151871:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317222.279708:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317223.175840:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317223.303268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317224.199350:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317224.327236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317225.095293:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317200/real 1767317200] req@ffff88c5c5728000 x1853163991624064/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317225 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317225.095327:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728000 x1853163991624064/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317225 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317225.095360:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317200/real 1767317200] req@ffff88c5c5729180 x1853163991624192/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317225 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317225.095373:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729180 x1853163991624192/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317225 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317225.095391:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317225.095421:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317225.095426:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991624064:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317225.095433:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317225.095438:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317225.095441:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991624192:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317225.095604:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317225.095609:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317225.095612:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317225.095618:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2399 00000100:00080000:1.0:1767317225.095627:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317225.095633:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317225.095661:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cfb80 x1853163991625088/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317225.095686:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317225.095689:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317225.095691:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317225.095695:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2399 00000100:00080000:1.0:1767317225.095698:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317225.095701:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317225.095708:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317225.095718:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ce300 x1853163991625216/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317225.095731:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0a800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:02000000:1.0:1767317225.095734:0:19117:0:(import.c:2080:ptlrpc_disconnect_and_idle_import()) lustre-OST0000-osc-ffff88c5c68bd000: disconnect after 20s idle 00000100:00100000:2.0:1767317225.097926:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991625088:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317225.098066:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991625216:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767317225.112144:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991625344:192.168.204.145@tcp:9:kworker.0 00000100:00080000:1.0:1767317225.124235:0:2485:0:(import.c:1990:ptlrpc_disconnect_idle_interpret()) @@@ inflight=1, refcount=3: rc = 0 req@ffff88c5c65cd880 x1853163991625344/t0(0) o9->lustre-OST0000-osc-ffff88c5c68bd000@192.168.204.145@tcp:28/4 lens 224/224 e 0 to 0 dl 1767317235 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767317225.124250:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5d1a0a800 lustre-OST0000_UUID: changing import state from CONNECTING to IDLE 00000100:00100000:1.0:1767317225.124272:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991625344:192.168.204.145@tcp:9:kworker.0 00010000:00010000:1.0:1767317225.223573:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317225.351307:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317226.119271:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317226.119280:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317226.119282:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317226.119284:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317226.247674:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317226.375234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317227.271317:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317227.399274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317228.295504:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317228.423332:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317229.319491:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317229.447218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317230.155575:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317230.155586:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317230.155590:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317230.155613:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317230.343310:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317230.471737:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317231.367439:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317231.499286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317232.395745:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317232.519579:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317233.416890:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317233.543178:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317234.439610:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317234.567773:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317235.335210:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317235.335221:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317235.335225:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317235.335228:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317235.463706:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317235.591246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317236.487889:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317236.615749:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317237.511411:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317237.639622:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317238.535329:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317238.663225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317239.559318:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317239.687800:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317240.457297:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317240.457308:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317240.457311:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317240.457314:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317240.584478:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317240.718272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317241.616982:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317241.735167:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317242.631345:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317242.759694:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317243.656651:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317243.783256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317244.680324:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317244.807237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317245.575893:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317245.575905:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317245.575908:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317245.575911:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317245.703295:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317245.837518:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317246.731272:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317246.855230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317247.751965:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317247.880635:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317248.780048:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317248.903230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317249.799305:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317249.927279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317250.695196:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317250.695207:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317250.695209:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317225/real 1767317225] req@ffff88c5c65cfb80 x1853163991625088/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317250 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767317250.695211:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317250.695235:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317250.695262:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cfb80 x1853163991625088/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317250 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317250.695273:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317225/real 1767317225] req@ffff88c5c65ce300 x1853163991625216/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317250 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317250.695284:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ce300 x1853163991625216/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317250 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317250.695301:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317250.695322:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317250.695327:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991625088:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317250.695333:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317250.695338:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317250.695342:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991625216:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317250.695537:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317250.695543:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317250.695546:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317250.695552:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2424 00000100:00080000:1.0:1767317250.695562:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317250.695569:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317250.695597:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cea00 x1853163991625472/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317250.695624:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317250.695627:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317250.695629:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317250.695633:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2424 00000100:00080000:1.0:1767317250.695636:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317250.695639:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317250.695648:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317250.695660:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ce300 x1853163991625600/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317250.695735:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991625472:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317250.695829:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991625600:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317250.823343:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317250.951559:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317251.850908:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317251.977599:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317252.871818:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317253.000914:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317253.895880:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317254.023211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317254.919567:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317255.048717:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317255.815420:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317255.815427:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317255.815429:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317255.815446:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317255.943889:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317256.071389:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317256.968957:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317257.095217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317257.995874:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317258.119245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317259.015918:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317259.143233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317260.039377:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317260.167311:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317260.936126:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317260.936144:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317260.936148:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317260.936151:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317261.063358:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317261.191368:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317262.087573:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317262.215247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317263.111727:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317263.239175:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317264.135630:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317264.263200:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317265.159343:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317265.287215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317266.056547:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317266.056576:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317266.056579:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317266.056583:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317266.183472:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317266.311283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317267.207328:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317267.335962:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317268.232005:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317268.359219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317269.255344:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317269.383339:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317270.279380:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317270.407248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317271.175288:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317271.175298:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317271.175301:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317271.175305:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317271.305355:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317271.432388:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317272.327346:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317272.455224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317273.351850:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317273.479395:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317274.376121:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317274.506638:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317275.271232:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317250/real 1767317250] req@ffff88c5c65cea00 x1853163991625472/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317275 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317275.271295:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cea00 x1853163991625472/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317275 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317275.271305:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317250/real 1767317250] req@ffff88c5c65ce300 x1853163991625600/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317275 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317275.271341:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ce300 x1853163991625600/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317275 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317275.271368:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317275.271396:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317275.271401:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991625472:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317275.271406:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317275.271412:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317275.271415:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991625600:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767317275.271574:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317275.271580:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767317275.271583:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317275.271588:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2449 00000100:00080000:3.0:1767317275.271597:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317275.271603:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767317275.271631:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b100 x1853163991625728/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767317275.271653:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317275.271656:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767317275.271658:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317275.271661:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2449 00000100:00080000:3.0:1767317275.271665:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317275.271668:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767317275.271676:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767317275.271687:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728700 x1853163991625856/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317275.271894:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991625728:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317275.271996:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991625856:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317275.399336:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317275.527212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317276.424892:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317276.551246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317277.456506:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317277.575471:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317278.471902:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317278.599312:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317279.499349:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317279.623365:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317280.392327:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317280.392339:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317280.392342:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317280.392346:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317280.519794:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317280.647227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317281.543346:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317281.671222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317282.567603:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317282.696613:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317283.591359:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317283.719246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317284.621420:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317284.743275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317285.511376:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317285.511387:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317285.511391:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317285.511394:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317285.639358:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317285.767270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317286.670584:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317286.791165:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317287.691215:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317287.820218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317288.713060:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317288.839366:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317289.735667:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317289.863891:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317290.631488:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317290.631498:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317290.631501:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317290.631505:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317290.759417:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317290.887227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317291.783873:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317291.911230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317292.810493:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317292.935238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317293.831520:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317293.959225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317294.855344:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317294.983215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317295.751224:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317295.751234:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317295.751237:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317295.751241:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317295.879508:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317296.007936:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317296.905479:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317297.031217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317297.928007:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317298.055589:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317298.952399:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317299.079226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317299.975372:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317300.103219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317300.871452:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317275/real 1767317275] req@ffff88c5c572b100 x1853163991625728/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317300 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317300.871537:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b100 x1853163991625728/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317300 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317300.871549:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317275/real 1767317275] req@ffff88c5c5728700 x1853163991625856/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317300 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317300.871563:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728700 x1853163991625856/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317300 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317300.871588:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317300.871600:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317300.871604:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991625728:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317300.871612:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317300.871617:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317300.871620:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991625856:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767317300.871710:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317300.871717:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767317300.871721:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317300.871726:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2474 00000100:00080000:3.0:1767317300.871738:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317300.871754:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767317300.871807:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728380 x1853163991625984/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767317300.871835:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317300.871839:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767317300.871841:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317300.871845:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2474 00000100:00080000:3.0:1767317300.871850:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317300.871853:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767317300.871866:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767317300.871880:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729880 x1853163991626112/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317300.871952:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991625984:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317300.872117:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991626112:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317300.999433:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317301.127262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317302.025016:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317302.151626:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317303.047547:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317303.175310:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317304.071421:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317304.199280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317305.097008:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317305.223345:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317305.991949:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317305.991965:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317305.991968:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317305.991972:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317306.120829:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317306.247250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317307.143905:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317307.271493:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317308.167581:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317308.295251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317309.191373:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317309.321622:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317310.215288:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317310.343267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317311.111214:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317311.111227:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317311.111229:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317311.111233:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317311.239328:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317311.367199:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317312.263731:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317312.391334:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317313.287524:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317313.415526:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317314.311324:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317314.439211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317315.336812:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317315.464800:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317316.231526:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317316.231537:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317316.231540:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317316.231544:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317316.359325:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317316.487248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317317.383363:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317317.512701:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317318.407811:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317318.538009:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317319.431623:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317319.559720:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317320.455337:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317320.583605:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317321.353305:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317321.353315:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317321.353318:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317321.353322:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317321.479697:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317321.607736:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317322.503623:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317322.631228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317323.533020:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317323.655317:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317324.551442:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317324.679360:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317325.447240:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317300/real 1767317300] req@ffff88c5c5728380 x1853163991625984/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317325 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317325.447278:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728380 x1853163991625984/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317325 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317325.447290:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317300/real 1767317300] req@ffff88c5c5729880 x1853163991626112/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317325 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317325.447341:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729880 x1853163991626112/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317325 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317325.447362:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317325.447401:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317325.447406:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991625984:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317325.447413:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317325.447418:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317325.447421:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991626112:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317325.447805:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317325.447812:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317325.447816:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317325.447822:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2500 00000100:00080000:1.0:1767317325.447833:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317325.447840:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317325.447885:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cfb80 x1853163991626240/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317325.447911:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317325.447914:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317325.447916:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317325.447921:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2500 00000100:00080000:1.0:1767317325.447925:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317325.447929:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317325.447939:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317325.447950:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd880 x1853163991626368/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317325.448075:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991626240:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317325.448215:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991626368:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317325.584148:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317325.703244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317326.599667:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317326.727231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317327.623350:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317327.751231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317328.648017:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317328.775306:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317329.672601:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317329.799297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317330.568297:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317330.568313:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317330.568316:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317330.568320:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317330.695315:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317330.823278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317331.719608:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317331.849191:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317332.743574:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317332.871871:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317333.767583:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317333.896774:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317334.795304:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317334.919373:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317335.687365:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317335.687376:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317335.687379:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317335.687383:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317335.817564:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317335.943215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317336.839709:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317336.967261:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317337.864538:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317337.991254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317338.887645:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317339.015459:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317339.911339:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317340.039801:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317340.807317:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317340.807326:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317340.807329:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317340.807354:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317340.935317:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317341.063991:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317341.959905:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317342.087798:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317342.986902:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317343.112509:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317344.008798:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317344.135169:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317345.035566:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317345.160021:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317345.929779:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317345.929791:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317345.929794:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317345.929798:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317346.055442:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317346.183364:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317347.079352:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317347.207227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317348.103752:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317348.231866:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317349.127521:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317349.255434:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317350.025150:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317325/real 1767317325] req@ffff88c5c65cfb80 x1853163991626240/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317350 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317350.025190:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cfb80 x1853163991626240/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317350 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317350.025201:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317325/real 1767317325] req@ffff88c5c65cd880 x1853163991626368/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317350 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317350.025239:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd880 x1853163991626368/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317350 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317350.025258:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317350.025287:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317350.025293:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991626240:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317350.025299:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317350.025304:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317350.025307:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991626368:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317350.026177:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317350.026187:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317350.026190:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317350.026198:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2524 00000100:00080000:1.0:1767317350.026209:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317350.026216:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317350.026252:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf100 x1853163991626496/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317350.026281:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317350.026285:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317350.026287:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317350.026291:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2524 00000100:00080000:1.0:1767317350.026294:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317350.026298:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317350.026307:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317350.026319:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf480 x1853163991626624/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317350.026397:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991626496:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317350.027723:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991626624:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317350.153850:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317350.282145:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317351.175481:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317351.303237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317352.199436:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317352.327378:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317353.223309:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317353.351231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317354.247323:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317354.375298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317355.143694:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317355.143706:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317355.143709:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317355.143712:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317355.271421:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317355.399251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317356.295802:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317356.423280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317357.319670:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317357.449343:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317358.347135:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317358.471841:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317359.368736:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317359.495217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317360.263273:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317360.263288:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317360.263291:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317360.263295:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317360.391343:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317360.519306:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317361.415320:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317361.544661:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317362.439938:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317362.567408:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317363.463666:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317363.592484:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317364.487802:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317364.615294:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317365.383332:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317365.383343:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317365.383346:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317365.383349:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317365.514136:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317365.639243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317366.535373:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317366.663233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317367.559350:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317367.688393:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317368.583450:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317368.711238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317369.610460:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317369.735623:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317370.503461:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317370.503474:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317370.503478:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317370.503482:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317370.635047:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317370.759216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317371.655737:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317371.783222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317372.679964:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317372.807229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317373.703922:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317373.831617:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317374.727421:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317374.858865:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317375.623259:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317350/real 1767317350] req@ffff88c5c65cf100 x1853163991626496/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317375 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317375.623300:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf100 x1853163991626496/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317375 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317375.623312:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317350/real 1767317350] req@ffff88c5c65cf480 x1853163991626624/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317375 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317375.623325:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf480 x1853163991626624/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317375 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317375.623345:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317375.623422:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317375.623427:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991626496:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317375.623434:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317375.623440:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317375.623443:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991626624:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767317375.624731:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317375.624740:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767317375.624745:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317375.624752:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2549 00000100:00080000:0.0:1767317375.624763:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317375.624769:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767317375.624829:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400e00 x1853163991626752/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317375.624860:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317375.624863:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767317375.624865:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317375.624869:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2549 00000100:00080000:0.0:1767317375.624874:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317375.624877:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767317375.624887:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767317375.624897:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402680 x1853163991626880/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317375.625350:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991626752:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317375.625476:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991626880:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317375.751488:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317375.879536:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317376.776321:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317376.903494:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317377.799850:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317377.927234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317378.823733:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317378.951215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317379.847461:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317379.975264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317380.745442:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317380.745453:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317380.745456:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317380.745460:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317380.871465:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317380.999227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317381.895397:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317382.023213:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317382.920058:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317383.047791:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317383.947233:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317384.071376:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317384.967321:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317385.095484:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317385.864312:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317385.864320:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317385.864324:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317385.864328:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317385.991532:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317386.119813:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317387.015570:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317387.143416:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317388.039406:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317388.167247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317389.063346:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317389.191255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317390.088728:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317390.218067:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317390.983318:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317390.983333:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317390.983336:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317390.983340:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317391.111427:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317391.239351:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317392.135535:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317392.263378:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317393.159648:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317393.287355:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317394.183315:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317394.311249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317395.209647:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317395.335229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317396.103649:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317396.103659:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317396.103662:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317396.103666:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317396.232699:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317396.359449:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317397.255275:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317397.383225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317398.286628:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317398.408628:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317399.303926:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317399.431224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317400.199270:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317375/real 1767317375] req@ffff88c5c6400e00 x1853163991626752/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317400 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317400.199317:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400e00 x1853163991626752/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317400 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317400.199327:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317375/real 1767317375] req@ffff88c5c6402680 x1853163991626880/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317400 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317400.199338:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402680 x1853163991626880/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317400 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317400.199379:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317400.199406:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317400.199410:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991626752:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317400.199416:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317400.199421:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317400.199424:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991626880:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767317400.199526:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317400.199531:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767317400.199534:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317400.199545:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2574 00000100:00080000:0.0:1767317400.199554:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317400.199559:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767317400.199585:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401500 x1853163991627008/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317400.199636:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991627008:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317400.199802:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317400.199806:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767317400.199808:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317400.199813:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2574 00000100:00080000:0.0:1767317400.199818:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317400.199822:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767317400.199832:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767317400.199847:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401180 x1853163991627136/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317400.204778:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991627136:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317400.328536:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317400.455220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317401.354369:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317401.479235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317402.375357:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317402.503250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317403.399470:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317403.527247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317404.423497:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317404.551264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317405.320299:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317405.320310:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317405.320313:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317405.320317:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317405.447333:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317405.576091:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317406.471409:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317406.599225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317407.496432:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317407.623256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317408.519868:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317408.647287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317409.545742:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317409.671406:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317410.439298:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317410.439312:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317410.439315:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317410.439318:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317410.569092:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317410.695495:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317411.591666:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317411.719733:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317412.616244:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317412.746120:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317413.639749:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317413.768228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317414.663408:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317414.791259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317415.559794:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317415.559810:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317415.559814:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317415.559818:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317415.689739:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317415.816397:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317416.711372:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317416.839259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317417.738277:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317417.868616:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317418.759713:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317418.887240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317419.799004:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317419.911308:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317420.679484:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317420.679496:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317420.679499:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317420.679502:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317420.807445:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317420.935346:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317421.831641:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317421.959558:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317422.855766:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317422.983687:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317423.879396:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317424.007257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317424.904697:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317425.031246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317425.799205:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317400/real 1767317400] req@ffff88c5c6401180 x1853163991627136/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317425 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317425.799234:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401180 x1853163991627136/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317425 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317425.799244:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317400/real 1767317400] req@ffff88c5c6401500 x1853163991627008/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317425 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317425.799257:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401500 x1853163991627008/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317425 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317425.799300:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317425.799328:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317425.799332:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991627136:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767317425.799336:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317425.799340:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317425.799342:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991627008:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767317425.799439:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317425.799445:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317425.799448:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317425.799459:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2599 00000100:00080000:1.0:1767317425.799469:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317425.799475:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317425.799518:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cc380 x1853163991627264/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317425.799582:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991627264:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767317425.801026:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317425.801033:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317425.801036:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317425.801043:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2599 00000100:00080000:1.0:1767317425.801052:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317425.801056:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317425.801069:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317425.801114:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cce00 x1853163991627392/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317425.802373:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991627392:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317425.932017:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317426.055257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317426.951412:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317427.079231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317427.975327:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317428.103210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317428.999323:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317429.127220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317430.023471:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317430.151424:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317430.920566:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317430.920584:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317430.920587:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317430.920590:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317431.047333:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317431.175228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317432.071409:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317432.199257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317433.098449:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317433.223259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317434.119270:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317434.247200:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317435.143286:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317435.271354:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317436.042582:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317436.042592:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317436.042595:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317436.042598:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317436.167334:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317436.295219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317437.191351:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317437.319355:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317438.215452:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317438.343842:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317439.239653:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317439.367598:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317440.263395:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317440.391374:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317441.161278:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317441.161290:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317441.161293:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317441.161305:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317441.291256:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317441.415247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317442.311517:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317442.439290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317443.337131:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317443.463467:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317444.359336:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317444.487643:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317445.384496:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317445.511243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317446.279249:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317446.279262:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317446.279265:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317446.279269:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317446.409210:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317446.535292:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317447.431323:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317447.560072:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317448.455540:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317448.583240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317449.479314:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317449.607234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767317450.376380:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317425/real 1767317425] req@ffff88c5c65cce00 x1853163991627392/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317450 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317450.376424:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cce00 x1853163991627392/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317450 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317450.376441:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317425/real 1767317425] req@ffff88c5c65cc380 x1853163991627264/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317450 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317450.376454:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cc380 x1853163991627264/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317450 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317450.376474:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317450.376505:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317450.376510:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991627392:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767317450.376517:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317450.376556:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317450.376561:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991627264:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767317450.376607:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317450.376619:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317450.376624:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317450.376635:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2624 00000100:00080000:1.0:1767317450.376646:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317450.376654:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317450.376694:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cca80 x1853163991627520/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317450.376737:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317450.376741:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317450.376743:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317450.376747:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2624 00000100:00080000:1.0:1767317450.376751:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317450.376755:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317450.376766:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317450.376779:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ced80 x1853163991627648/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317450.376879:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991627520:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317450.376997:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991627648:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317450.504081:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317450.631705:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317451.535166:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317451.655400:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317452.555595:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317452.679227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317453.575318:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317453.703228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317454.599399:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317454.730233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317455.495261:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317455.495299:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317455.495303:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317455.495307:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317455.623645:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317455.756069:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317456.649279:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317456.775860:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317457.671571:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317457.799313:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317458.696310:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317458.823252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317459.719323:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317459.848024:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317460.615994:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317460.616005:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317460.616007:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317460.616011:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317460.745248:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317460.871264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317461.769592:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317461.896741:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317462.795371:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317462.924936:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317463.817485:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317463.943242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317464.839302:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317464.967269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317465.735954:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317465.735962:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317465.735965:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317465.735969:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317465.863325:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317465.991217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317466.887485:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317467.015222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317467.911319:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317468.039620:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317468.936193:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317469.065417:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317469.964759:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317470.087217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317470.855946:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317470.855956:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317470.855959:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317470.855962:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317470.983452:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317471.111423:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317472.008814:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317472.136245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317473.031686:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317473.159234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317474.055712:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317474.184429:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317475.079973:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317475.207960:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317475.975326:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317475.975338:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317475.975341:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317475.975345:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767317475.975934:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317450/real 1767317450] req@ffff88c5c65cca80 x1853163991627520/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317475 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317475.975968:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cca80 x1853163991627520/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317475 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317475.975979:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317450/real 1767317450] req@ffff88c5c65ced80 x1853163991627648/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317475 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317475.976013:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ced80 x1853163991627648/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317475 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317475.976033:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317475.976056:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317475.976060:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991627520:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317475.976067:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317475.976073:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317475.976076:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991627648:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767317475.976378:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317475.976384:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767317475.976387:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317475.976392:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2649 00000100:00080000:3.0:1767317475.976402:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317475.976408:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767317475.976440:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728a80 x1853163991627776/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767317475.976474:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317475.976478:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767317475.976480:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317475.976484:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2649 00000100:00080000:3.0:1767317475.976488:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317475.976493:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767317475.976501:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767317475.976515:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728e00 x1853163991627904/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317475.976583:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991627776:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317475.976696:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991627904:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317476.103390:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317476.231230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317477.127308:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317477.255277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317478.152022:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317478.279293:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317479.176488:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317479.304123:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317480.199971:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317480.327370:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317481.095337:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317481.095345:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317481.095348:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317481.095350:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317481.225711:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317481.357617:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317482.247530:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317482.375278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317483.272912:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317483.399272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317484.296450:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317484.427067:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317485.321511:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317485.447279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317486.215397:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317486.215413:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317486.215417:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317486.215421:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317486.343619:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317486.471490:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317487.367568:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317487.495517:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317488.395211:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317488.519254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317489.415514:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317489.543915:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317490.439563:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317490.567201:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317491.335563:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317491.335573:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317491.335576:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317491.335580:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317491.464380:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317491.591224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317492.487477:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317492.615239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317493.511637:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317493.639207:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317494.535465:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317494.663243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317495.560543:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317495.687265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317496.455644:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317496.455653:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317496.455657:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317496.455660:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317496.585593:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317496.711240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317497.612991:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317497.736923:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317498.633984:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317498.759278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317499.656299:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317499.783585:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767317500.551292:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317475/real 1767317475] req@ffff88c5c5728a80 x1853163991627776/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317500 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317500.551328:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728a80 x1853163991627776/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317500 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317500.551340:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317475/real 1767317475] req@ffff88c5c5728e00 x1853163991627904/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317500 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317500.551351:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728e00 x1853163991627904/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317500 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317500.551372:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317500.551406:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317500.551411:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991627776:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767317500.551418:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317500.551425:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317500.551428:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991627904:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317500.551543:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317500.551550:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317500.551554:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317500.551560:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2675 00000100:00080000:1.0:1767317500.551570:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317500.551578:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317500.551617:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cdf80 x1853163991628032/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317500.551645:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317500.551667:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317500.551669:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317500.551674:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2675 00000100:00080000:1.0:1767317500.551678:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317500.551682:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317500.551692:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317500.551706:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf800 x1853163991628160/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767317500.551784:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991628032:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767317500.551891:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991628160:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317500.680471:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317500.807220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317501.706637:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317501.831224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317502.727350:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317502.859723:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317503.751457:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317503.879243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317504.776004:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317504.903217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317505.674358:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317505.674369:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317505.674372:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317505.674375:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317505.806661:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317505.927301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317506.823489:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317506.951672:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317507.847775:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317507.975236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317508.871598:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317508.999354:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317509.895414:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317510.027692:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317510.791225:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317510.791237:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317510.791240:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317510.791243:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317510.919278:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317511.048355:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317511.943795:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317512.072791:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317512.967558:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317513.095201:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317513.991610:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317514.119344:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317515.015276:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317515.143338:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317515.911338:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317515.911362:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317515.911365:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317515.911368:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317516.042003:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317516.167225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317517.063410:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317517.191229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317518.087410:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317518.215225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317519.111359:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317519.239327:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317520.135326:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317520.263241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317521.031344:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317521.031355:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317521.031358:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317521.031361:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317521.159314:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317521.288994:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317522.185158:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317522.311217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317523.207481:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317523.335212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317524.233394:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317524.359652:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317525.127212:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317500/real 1767317500] req@ffff88c5c65cdf80 x1853163991628032/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317525 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317525.127247:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cdf80 x1853163991628032/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317525 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317525.127258:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317500/real 1767317500] req@ffff88c5c65cf800 x1853163991628160/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317525 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317525.127280:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf800 x1853163991628160/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317525 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317525.127298:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317525.127325:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317525.127330:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991628032:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317525.127336:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317525.127340:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317525.127343:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991628160:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767317525.127576:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317525.127582:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767317525.127585:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317525.127590:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2699 00000100:00080000:3.0:1767317525.127600:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317525.127606:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767317525.127647:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728e00 x1853163991628288/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767317525.127670:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317525.127674:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767317525.127696:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317525.127701:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2699 00000100:00080000:3.0:1767317525.127706:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317525.127710:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767317525.127719:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767317525.127731:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728a80 x1853163991628416/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317525.127798:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991628288:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317525.127899:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991628416:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317525.255337:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317525.384302:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317526.279484:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317526.407291:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317527.303428:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317527.431210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317528.328203:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317528.455212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317529.351921:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317529.479229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317530.247635:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317530.247648:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317530.247651:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317530.247654:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317530.375596:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317530.503292:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317531.406002:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317531.527841:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317532.423349:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317532.551221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317533.447353:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317533.575219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317534.472431:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317534.599900:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317535.367325:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317535.367335:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317535.367338:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317535.367341:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317535.495689:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317535.623285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317536.519461:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317536.647210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317537.546538:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317537.671218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317538.567304:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317538.701587:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317539.592226:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317539.720842:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317540.487299:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317540.487309:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317540.487313:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317540.487317:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317540.615351:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317540.743281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317541.639558:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317541.769286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317542.663339:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317542.791238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317543.687388:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317543.815447:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317544.716464:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317544.839681:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317545.607858:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317545.607869:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317545.607873:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317545.607876:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317545.735352:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317545.863517:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317546.761487:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317546.889667:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317547.783384:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317547.911251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317548.810209:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317548.935215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317549.835377:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317549.960680:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317550.727256:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317525/real 1767317525] req@ffff88c5c5728e00 x1853163991628288/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317550 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317550.727288:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728e00 x1853163991628288/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317550 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317550.727299:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317525/real 1767317525] req@ffff88c5c5728a80 x1853163991628416/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317550 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317550.727311:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728a80 x1853163991628416/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317550 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317550.727329:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317550.727355:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317550.727360:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991628288:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317550.727367:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317550.727372:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317550.727375:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991628416:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767317550.727471:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317550.727476:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767317550.727480:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317550.727485:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2724 00000100:00080000:0.0:1767317550.727493:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317550.727500:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767317550.727532:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403800 x1853163991628544/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317550.727556:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317550.727559:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767317550.727561:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317550.727565:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2724 00000100:00080000:0.0:1767317550.727568:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317550.727571:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767317550.727580:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767317550.727591:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400700 x1853163991628672/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767317550.727653:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991628544:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317550.727742:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991628672:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317550.855361:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317550.983266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317551.879391:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317552.008091:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317552.907135:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317553.031259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317553.928129:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317554.055253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317554.951357:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317555.079242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317555.847345:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317555.847356:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317555.847359:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317555.847362:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317555.977421:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317556.103443:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317557.000343:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317557.127276:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317558.023677:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317558.151346:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317559.050333:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317559.175226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317560.071876:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317560.199839:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317560.967280:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317560.967293:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317560.967318:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317560.967322:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317561.100748:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317561.223744:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317562.119711:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317562.247215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317563.143375:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317563.271751:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317564.169043:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317564.295228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317565.191322:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317565.320055:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317566.087255:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317566.087335:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317566.087338:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317566.087342:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317566.215879:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317566.343634:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317567.239377:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317567.367509:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317568.263842:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317568.391241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317569.287546:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317569.415382:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317570.311693:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317570.439315:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317571.207952:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317571.207967:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317571.207993:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317571.207997:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317571.339606:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317571.463257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317572.365891:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317572.487266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317573.384192:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317573.512281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317574.407380:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317574.537993:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767317575.303276:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317550/real 1767317550] req@ffff88c5c6403800 x1853163991628544/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317575 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317575.303323:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403800 x1853163991628544/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317575 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317575.303337:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317550/real 1767317550] req@ffff88c5c6400700 x1853163991628672/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317575 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767317575.303351:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400700 x1853163991628672/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317575 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767317575.303375:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317575.303464:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317575.303469:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991628544:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317575.303477:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767317575.304893:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317575.304930:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317575.304933:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317575.304943:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2749 00000100:00080000:1.0:1767317575.304956:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317575.304963:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317575.306958:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cc000 x1853163991628800/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317575.306985:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317575.306990:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317575.306992:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317575.306998:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2749 00000100:00080000:1.0:1767317575.307008:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317575.307092:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317575.307120:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317575.307139:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd500 x1853163991628928/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767317575.307276:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767317575.307284:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991628672:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767317575.307334:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991628800:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767317575.309322:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991628928:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317575.433263:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317575.559265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317576.455420:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317576.583355:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317577.482297:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317577.607280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317578.503632:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317578.631228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317579.534905:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317579.656819:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317580.424814:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317580.424824:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317580.424828:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317580.424831:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317580.551613:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317580.679226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317581.578321:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317581.704873:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317582.599363:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317582.727229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317583.623659:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317583.751207:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317584.647331:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317584.775192:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317585.543540:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317585.543551:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317585.543554:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317585.543557:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317585.675535:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317585.799229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317586.695520:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317586.823287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317587.719383:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317587.847233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317588.743343:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317588.875165:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317589.769560:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317589.895862:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317590.665005:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317590.665015:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317590.665018:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317590.665021:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317590.791344:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317590.919413:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317591.815343:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317591.943219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317592.839301:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317592.968364:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317593.863516:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317593.991341:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317594.890896:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317595.015297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317595.784627:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317595.784637:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317595.784640:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317595.784644:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317595.911303:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317596.039229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317596.935617:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317597.063257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317597.959331:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317598.087195:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317598.983304:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317599.111173:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317600.007448:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317600.135215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317600.903233:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317600.903245:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317600.903250:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317600.903254:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317600.903414:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317575/real 1767317575] req@ffff88c5c65cc000 x1853163991628800/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317600 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317600.903447:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cc000 x1853163991628800/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317600 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317600.903458:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317575/real 1767317575] req@ffff88c5c65cd500 x1853163991628928/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317600 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317600.903560:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd500 x1853163991628928/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317600 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317600.903581:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317600.903601:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317600.903605:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991628800:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317600.903611:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317600.903616:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00080000:1.0:1767317600.903630:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317600.903635:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317600.903638:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317600.903644:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2774 00000100:00080000:1.0:1767317600.903654:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317600.903676:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767317600.903689:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991628928:192.168.204.145@tcp:38:kworker.0 00000100:00100000:1.0:1767317600.903709:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ce680 x1853163991629056/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317600.904187:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317600.904190:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317600.904192:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317600.904198:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2774 00000100:00080000:1.0:1767317600.904203:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317600.904207:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317600.904220:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317600.904239:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd180 x1853163991629184/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317600.904399:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991629056:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317600.904568:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991629184:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317601.031424:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317601.159219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317602.057032:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317602.183275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317603.080319:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317603.207454:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317604.103688:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317604.231273:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317605.127461:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317605.255282:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317606.023338:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317606.023352:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317606.023356:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317606.023359:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317606.151407:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317606.279305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317607.176490:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317607.304377:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317608.202344:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317608.327411:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317609.223373:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317609.351239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317610.247375:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317610.375785:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317611.149440:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317611.149453:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317611.149456:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317611.149460:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317611.271442:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317611.399401:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317612.295911:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317612.423786:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317613.321371:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317613.447589:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317614.343312:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317614.471229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317615.368077:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317615.495226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317616.263371:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317616.263378:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317616.263381:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317616.263383:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317616.391322:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317616.519244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317617.415666:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317617.543228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317618.443065:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317618.567480:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317619.465701:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317619.591222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317620.487288:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317620.615279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317621.383230:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317621.383239:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317621.383242:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317621.383264:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317621.511390:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317621.639226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317622.536278:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317622.663255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317623.563622:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317623.687268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317624.583487:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317624.711230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317625.479317:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317600/real 1767317600] req@ffff88c5c65ce680 x1853163991629056/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317625 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317625.479350:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ce680 x1853163991629056/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317625 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317625.479362:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317600/real 1767317600] req@ffff88c5c65cd180 x1853163991629184/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317625 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317625.479374:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd180 x1853163991629184/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317625 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317625.479391:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317625.479448:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317625.479453:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991629056:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317625.479460:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317625.479569:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317625.479574:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767317625.479578:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317625.479583:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2800 00000100:00080000:3.0:1767317625.479593:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317625.479600:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767317625.479635:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b480 x1853163991629312/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767317625.479649:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317625.479652:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767317625.479654:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317625.479658:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2800 00000100:00080000:3.0:1767317625.479661:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317625.479681:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767317625.479689:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767317625.479701:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572a680 x1853163991629440/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317625.479788:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317625.479793:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991629184:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767317625.479819:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991629312:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317625.479936:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991629440:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317625.608031:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317625.735269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317626.631338:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317626.759209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317627.655467:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317627.785673:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317628.679314:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317628.807220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317629.703383:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317629.831223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317630.603374:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317630.603385:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317630.603388:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317630.603392:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317630.729020:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317630.855443:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317631.752067:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317631.879347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317632.776386:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317632.903383:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317633.801986:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317633.927558:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317634.823582:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317634.951792:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317635.720893:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317635.720904:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317635.720908:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317635.720931:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317635.847779:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317635.977171:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317636.872692:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317636.999267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317637.895472:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317638.023217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317638.922141:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317639.047239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317639.943629:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317640.071320:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317640.840519:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317640.840545:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317640.840548:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317640.840551:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317640.967394:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317641.095339:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317641.991593:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317642.119224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317643.015546:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317643.143242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317644.040035:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317644.167429:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317645.068054:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317645.191230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317645.959494:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317645.959504:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317645.959506:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317645.959510:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317646.087561:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317646.215235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317647.112282:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317647.239628:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317648.135798:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317648.263223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317649.163503:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317649.287695:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317650.055243:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317625/real 1767317625] req@ffff88c5c572b480 x1853163991629312/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317650 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317650.055274:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b480 x1853163991629312/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317650 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317650.055285:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317625/real 1767317625] req@ffff88c5c572a680 x1853163991629440/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317650 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317650.055296:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572a680 x1853163991629440/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317650 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317650.055314:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317650.055343:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317650.055347:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991629312:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317650.055353:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317650.055359:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317650.055362:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991629440:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767317650.055535:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317650.055541:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767317650.055545:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317650.055550:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2824 00000100:00080000:3.0:1767317650.055561:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317650.055568:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767317650.055620:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572bb80 x1853163991629568/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767317650.055647:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317650.055650:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767317650.055652:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317650.055655:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2824 00000100:00080000:3.0:1767317650.055660:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317650.055663:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767317650.055673:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767317650.055686:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572a680 x1853163991629696/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317650.055821:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991629568:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317650.055914:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991629696:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317650.186501:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317650.311248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317651.207330:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317651.335428:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317652.231418:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317652.359456:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317653.255290:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317653.383496:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317654.279299:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317654.407301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317655.175679:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317655.175690:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317655.175692:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317655.175696:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317655.303329:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317655.431217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317656.327321:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317656.457633:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317657.354129:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317657.479217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317658.375431:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317658.503224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317659.399244:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317659.527233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317660.295428:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317660.295439:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317660.295442:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317660.295445:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317660.425599:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317660.551578:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317661.447655:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317661.575922:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317662.471466:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317662.599278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317663.495997:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317663.623279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317664.519671:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317664.649363:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317665.415224:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317665.415235:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317665.415254:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317665.415259:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317665.543384:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317665.672476:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317666.567473:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317666.695217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317667.591324:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317667.719347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317668.615318:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317668.743369:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317669.640631:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317669.767233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317670.535206:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317670.535216:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317670.535219:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317670.535222:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317670.663298:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317670.792960:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317671.687741:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317671.815337:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317672.711416:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317672.840021:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317673.735878:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317673.863222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317674.759360:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317674.887902:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317675.655326:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317675.655337:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317675.655340:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317675.655343:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317675.655671:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317650/real 1767317650] req@ffff88c5c572bb80 x1853163991629568/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317675 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317675.655701:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572bb80 x1853163991629568/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317675 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317675.655734:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317650/real 1767317650] req@ffff88c5c572a680 x1853163991629696/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317675 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317675.655748:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572a680 x1853163991629696/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317675 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317675.655767:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317675.655824:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317675.655829:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991629568:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317675.655835:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317675.656209:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317675.656221:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767317675.656224:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317675.656239:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2849 00000100:00080000:2.0:1767317675.656248:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317675.656253:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767317675.656303:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5180 x1853163991629824/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767317675.656316:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317675.656319:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767317675.656321:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317675.656324:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2849 00000100:00080000:2.0:1767317675.656328:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317675.656332:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767317675.656341:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767317675.656352:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4700 x1853163991629952/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317675.656881:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317675.656887:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991629696:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767317675.656928:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991629824:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317675.657034:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991629952:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317675.783380:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317675.911208:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317676.807709:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317676.935266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317677.831418:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317677.959244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317678.857079:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317678.983251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317679.879569:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317680.007212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317680.775471:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317680.775573:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317680.775576:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317680.775580:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317680.903340:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317681.033127:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317681.927551:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317682.055457:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317682.951602:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317683.083954:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317683.976582:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317684.105320:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317685.000741:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317685.127231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317685.895667:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317685.895678:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317685.895681:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317685.895684:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317686.024112:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317686.151227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317687.047336:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317687.175241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317688.071808:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317688.199414:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317689.095284:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317689.223250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317690.119464:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317690.248577:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317691.015997:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317691.016007:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317691.016087:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317691.018959:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317691.143362:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317691.271652:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317692.167300:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317692.295205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317693.191334:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317693.319277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317694.215988:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317694.343227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317695.240370:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317695.368490:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317696.136826:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317696.136837:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317696.136840:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317696.136844:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317696.263434:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317696.391219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317697.290222:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317697.415271:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317698.311449:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317698.443571:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317699.335320:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317699.463240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767317700.232085:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317675/real 1767317675] req@ffff88c5c57d5180 x1853163991629824/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317700 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317700.232152:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5180 x1853163991629824/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317700 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317700.232163:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317675/real 1767317675] req@ffff88c5c57d4700 x1853163991629952/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317700 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317700.232176:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4700 x1853163991629952/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317700 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317700.232193:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317700.232221:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317700.232225:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991629824:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767317700.232231:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317700.232255:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317700.232259:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991629952:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317700.232529:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317700.232536:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317700.232539:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317700.232545:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2874 00000100:00080000:1.0:1767317700.232555:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317700.232598:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317700.232629:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cdc00 x1853163991630080/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317700.232658:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317700.232662:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317700.232664:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317700.232667:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2874 00000100:00080000:1.0:1767317700.232671:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317700.232675:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317700.232692:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317700.232706:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd180 x1853163991630208/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767317700.232932:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991630080:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767317700.233031:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991630208:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317700.360670:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317700.487233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317701.383913:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317701.513349:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317702.410417:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317702.535267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317703.434749:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317703.559322:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317704.455381:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317704.583819:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317705.351381:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317705.351392:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317705.351395:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317705.351398:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317705.479571:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317705.607234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317706.506556:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317706.631574:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317707.533044:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317707.655272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317708.551353:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317708.680868:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317709.575261:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317709.703458:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317710.477916:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317710.477928:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317710.477931:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317710.477934:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317710.599360:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317710.727506:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317711.623602:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317711.753139:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317712.647873:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317712.775218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317713.672081:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317713.799498:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317714.701386:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317714.823608:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317715.591911:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317715.591921:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317715.591924:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317715.591928:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317715.719411:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317715.847615:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317716.745051:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317716.871281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317717.771259:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317717.895288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317718.791334:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317718.931379:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317719.815345:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317719.943442:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317720.712194:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317720.712221:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317720.712224:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317720.712227:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317720.844791:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317720.967218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317721.863450:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317721.999798:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317722.891696:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317723.015275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317723.911385:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317724.039220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317724.935492:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317725.063179:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317725.831981:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317700/real 1767317700] req@ffff88c5c65cdc00 x1853163991630080/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317725 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317725.832052:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cdc00 x1853163991630080/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317725 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317725.832064:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317700/real 1767317700] req@ffff88c5c65cd180 x1853163991630208/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317725 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317725.832077:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd180 x1853163991630208/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317725 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317725.832127:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317725.832185:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317725.832190:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991630080:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317725.832204:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767317725.842410:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317725.842422:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767317725.842426:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317725.842437:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2899 00000100:00080000:2.0:1767317725.842449:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317725.842456:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767317725.842493:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6d80 x1853163991630336/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767317725.842512:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317725.842515:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767317725.842517:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317725.842521:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2899 00000100:00080000:2.0:1767317725.842524:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317725.842527:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767317725.842535:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767317725.842547:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4700 x1853163991630464/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317725.847538:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317725.847551:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991630208:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767317725.847599:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991630336:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317725.847732:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991630464:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317725.959399:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317726.087375:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317726.985388:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317727.111819:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317728.007321:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317728.135214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317729.031230:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317729.159227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317730.055419:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317730.184181:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317730.951474:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317730.951485:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317730.951489:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317730.951492:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317731.079693:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317731.207269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317732.103945:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317732.231625:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317733.127369:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317733.258229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317734.151317:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317734.279208:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317735.180275:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317735.303246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317736.071249:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317736.071277:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317736.071281:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317736.071284:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317736.201880:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317736.327246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317737.223331:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317737.351233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317738.248072:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317738.375225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317739.271487:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317739.400133:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317740.295377:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317740.423536:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317741.192882:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317741.192892:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317741.192894:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317741.192898:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317741.319423:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317741.447236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317742.349826:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317742.472831:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317743.367932:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317743.495228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317744.391543:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317744.519282:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317745.415389:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317745.543615:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317746.318954:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317746.318965:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317746.318968:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317746.318972:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317746.439354:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317746.567277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317747.463307:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317747.591459:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317748.487352:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317748.615217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317749.511716:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317749.639362:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317750.407358:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317725/real 1767317725] req@ffff88c5c57d6d80 x1853163991630336/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317750 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317750.407401:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6d80 x1853163991630336/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317750 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317750.407414:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317725/real 1767317725] req@ffff88c5c57d4700 x1853163991630464/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317750 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317750.407426:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4700 x1853163991630464/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317750 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317750.407446:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317750.407479:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317750.407483:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991630336:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317750.407489:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317750.407515:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317750.407519:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991630464:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767317750.408684:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317750.408704:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767317750.408708:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317750.408718:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2925 00000100:00080000:2.0:1767317750.408730:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317750.408744:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767317750.408782:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5180 x1853163991630592/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767317750.408813:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317750.408816:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767317750.408818:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317750.408822:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2925 00000100:00080000:2.0:1767317750.408826:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317750.408830:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767317750.408839:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767317750.408851:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7b80 x1853163991630720/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317750.408975:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991630592:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317750.409123:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991630720:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317750.536958:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317750.663370:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317751.559621:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317751.687297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317752.583638:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317752.711337:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317753.607824:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317753.737786:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317754.632166:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317754.759222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317755.527266:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317755.527281:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317755.527284:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317755.527288:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317755.655309:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317755.783651:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317756.680000:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317756.807571:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317757.705948:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317757.831224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317758.727287:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317758.855700:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317759.754776:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317759.879229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317760.647431:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317760.647459:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317760.647463:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317760.647466:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317760.785424:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317760.903225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317761.799405:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317761.928152:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317762.823374:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317762.951226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317763.851124:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317763.975999:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317764.871369:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317764.999216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317765.771574:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317765.771583:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317765.771586:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317765.771589:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317765.896420:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317766.023219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317766.922178:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317767.047230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317767.944278:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317768.071266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317768.967442:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317769.096440:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317769.991730:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317770.119273:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317770.898508:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317770.898538:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317770.898541:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317770.898544:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317771.015519:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317771.143230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317772.042371:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317772.167709:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317773.066351:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317773.191248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317774.087512:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317774.215318:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317775.111962:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317775.239276:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317776.007385:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317776.007396:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317776.007399:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317776.007403:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317776.007565:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317750/real 1767317750] req@ffff88c5c57d5180 x1853163991630592/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317775 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317776.007607:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5180 x1853163991630592/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317775 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317776.007617:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317750/real 1767317750] req@ffff88c5c57d7b80 x1853163991630720/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317775 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317776.007627:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7b80 x1853163991630720/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317775 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317776.007646:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317776.007666:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317776.007670:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991630592:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767317776.007676:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317776.007681:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317776.007684:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991630720:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767317776.007866:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317776.007871:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767317776.007875:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317776.007881:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2949 00000100:00080000:0.0:1767317776.007891:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317776.007898:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767317776.007934:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403480 x1853163991630848/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317776.007957:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317776.007961:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767317776.007962:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317776.007967:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2949 00000100:00080000:0.0:1767317776.007971:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317776.007974:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767317776.007989:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767317776.008002:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401f80 x1853163991630976/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767317776.008158:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991630848:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767317776.008271:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991630976:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317776.136409:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317776.263498:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317777.160555:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317777.289460:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317778.190970:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317778.313445:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317779.209083:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317779.335235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317780.231318:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317780.359668:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317781.127943:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317781.127954:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317781.127957:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317781.127960:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317781.256965:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317781.383281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317782.280827:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317782.407234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317783.304428:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317783.431588:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317784.328710:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317784.455336:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317785.351374:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317785.482194:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317786.247272:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317786.247283:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317786.247286:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317786.247310:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317786.375422:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317786.503220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317787.399681:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317787.527235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317788.426385:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317788.553396:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317789.454034:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317789.575417:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317790.471493:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317790.600771:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317791.368997:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317791.369012:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317791.369015:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317791.369019:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317791.495312:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317791.623351:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317792.520529:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317792.647280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317793.545806:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317793.671984:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317794.568778:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317794.697872:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317795.591879:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317795.719260:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317796.492745:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317796.492755:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317796.492757:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317796.492761:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317796.615310:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317796.743421:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317797.639938:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317797.767326:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317798.665776:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317798.791297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317799.687376:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317799.815639:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317800.711282:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317800.840592:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317801.607221:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1767317801.607225:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317776/real 1767317776] req@ffff88c5c6403480 x1853163991630848/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317801 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767317801.607233:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317801.607238:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317801.607244:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767317801.607263:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403480 x1853163991630848/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317801 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317801.607275:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317776/real 1767317776] req@ffff88c5c6401f80 x1853163991630976/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317801 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317801.607311:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401f80 x1853163991630976/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317801 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317801.607330:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317801.607354:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317801.607359:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991630848:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767317801.607365:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317801.607370:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317801.607373:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991630976:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317801.607398:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317801.607404:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317801.607407:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317801.607414:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 2975 00000100:00080000:1.0:1767317801.607424:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317801.607432:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317801.607478:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ce680 x1853163991631104/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317801.607506:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317801.607510:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317801.607511:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317801.607516:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 2975 00000100:00080000:1.0:1767317801.607521:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317801.607525:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317801.607535:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767317801.607540:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991631104:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767317801.607551:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd500 x1853163991631232/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767317801.607684:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991631232:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317801.735375:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317801.863264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317802.759325:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317802.887226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317803.783390:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317803.911233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317804.809088:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317804.935251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317805.832147:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317805.959255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317806.727705:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317806.727717:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317806.727720:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317806.727723:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317806.855409:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317806.983419:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317807.879337:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317808.007228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317808.907838:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317809.031310:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317809.930169:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317810.055222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317810.951310:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317811.079298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317811.847483:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317811.847494:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317811.847497:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317811.847501:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317811.975367:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317812.103239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317812.999319:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317813.127356:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317814.023354:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317814.151220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317815.047311:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317815.175220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317816.071396:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317816.203498:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317816.969665:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317816.969707:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317816.969710:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317816.969714:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317817.096955:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317817.223750:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317818.122557:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767317818.247281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317819.143870:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317819.277244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317820.168244:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317820.298028:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317821.191463:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317821.319345:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317822.087593:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317822.087604:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317822.087606:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317822.087609:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317822.215528:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317822.343272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317823.245704:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317823.367602:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317824.263593:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317824.391275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317825.287567:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317825.416628:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767317826.183775:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317801/real 1767317801] req@ffff88c5c65cd500 x1853163991631232/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317826 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317826.183822:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd500 x1853163991631232/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317826 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317826.183834:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317801/real 1767317801] req@ffff88c5c65ce680 x1853163991631104/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317826 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317826.183846:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ce680 x1853163991631104/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317826 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317826.183870:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317826.183902:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317826.183907:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991631232:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767317826.183913:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317826.183918:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317826.183921:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991631104:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317826.185258:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317826.185271:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767317826.185276:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317826.185286:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3000 00000100:00080000:0.0:1767317826.185302:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317826.185310:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767317826.185361:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402a00 x1853163991631360/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767317826.185402:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317826.185406:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767317826.185408:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767317826.185413:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3000 00000100:00080000:0.0:1767317826.185417:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767317826.185421:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767317826.185432:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767317826.185446:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402300 x1853163991631488/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767317826.185553:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991631360:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767317826.185706:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991631488:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317826.311374:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317826.439285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317827.339418:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317827.463283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317828.359409:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317828.487665:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317829.383829:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317829.511850:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317830.407323:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317830.535210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317831.303365:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317831.303375:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317831.303377:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317831.303381:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317831.432403:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317831.559442:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317832.455361:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317832.583372:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317833.479578:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317833.607222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317834.505929:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317834.632213:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317835.527388:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317835.655225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317836.423277:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317836.423288:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317836.423291:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317836.423295:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317836.551895:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317836.679222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317837.575332:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317837.703659:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317838.602044:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317838.728166:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317839.624133:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317839.751209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317840.647511:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317840.775219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317841.543539:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317841.543562:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317841.543565:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317841.543569:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317841.671352:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317841.799244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317842.695319:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317842.824048:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317843.719317:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317843.847483:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317844.743742:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317844.873068:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317845.767892:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317845.895207:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317846.664383:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317846.664392:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317846.664394:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317846.664397:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317846.791473:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317846.919195:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317847.816073:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317847.943200:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317848.839458:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317848.967644:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317849.863262:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317849.991219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317850.887264:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317851.015952:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317851.783255:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317826/real 1767317826] req@ffff88c5c6402a00 x1853163991631360/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317851 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317851.783292:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402a00 x1853163991631360/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317851 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317851.783331:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317826/real 1767317826] req@ffff88c5c6402300 x1853163991631488/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317851 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317851.783344:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402300 x1853163991631488/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317851 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317851.783363:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317851.783385:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317851.783389:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991631360:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317851.783395:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317851.783401:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317851.783404:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991631488:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767317851.783508:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317851.783513:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767317851.783516:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317851.783521:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3025 00000100:00080000:3.0:1767317851.783539:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317851.783545:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767317851.783581:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b480 x1853163991631616/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767317851.783606:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317851.783609:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767317851.783610:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767317851.783614:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3025 00000100:00080000:3.0:1767317851.783617:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767317851.783620:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767317851.783630:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767317851.783640:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728a80 x1853163991631744/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317851.783713:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991631616:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317851.783849:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991631744:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317851.911778:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317852.040879:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317852.935733:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317853.063154:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317853.962065:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317854.087234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317854.983354:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317855.111336:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317856.012089:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317856.135232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317856.903242:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317856.903254:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317856.903257:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317856.903259:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317857.033524:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317857.160343:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317858.055784:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317858.183387:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317859.084142:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317859.207326:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317860.103553:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317860.231336:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317861.127317:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317861.255229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317862.023372:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317862.023382:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317862.023385:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317862.023406:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317862.152503:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317862.279222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317863.177613:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317863.303202:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317864.199348:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317864.327196:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317865.223352:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317865.351322:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317866.247294:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317866.375250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317867.143951:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317867.143962:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317867.143965:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317867.143968:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317867.271660:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317867.400689:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317868.295308:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317868.423213:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317869.321876:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317869.447235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317870.343385:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317870.471217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317871.367413:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317871.495240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317872.263184:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317872.263195:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317872.263198:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317872.264071:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317872.391650:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317872.519544:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317873.416929:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317873.543216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317874.441168:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317874.567245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317875.464741:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317875.591299:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767317876.362604:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317851/real 1767317851] req@ffff88c5c572b480 x1853163991631616/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317876 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317876.362641:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b480 x1853163991631616/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317876 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317876.362653:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317851/real 1767317851] req@ffff88c5c5728a80 x1853163991631744/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317876 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767317876.362689:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728a80 x1853163991631744/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317876 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767317876.362706:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317876.362732:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317876.362737:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991631616:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767317876.362743:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767317876.362748:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767317876.362751:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991631744:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767317876.362853:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317876.362860:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767317876.362863:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317876.362869:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3050 00000100:00080000:2.0:1767317876.362878:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317876.362884:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767317876.362922:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5880 x1853163991631872/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767317876.362951:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317876.362954:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767317876.362956:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317876.362960:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3050 00000100:00080000:2.0:1767317876.362964:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317876.362968:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767317876.362977:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767317876.362989:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4380 x1853163991632000/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767317876.363064:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991631872:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767317876.363190:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991632000:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767317876.487363:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317876.615828:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317877.511375:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317877.639298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317878.535334:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317878.663354:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317879.559441:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317879.687272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317880.587873:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317880.713604:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317881.480180:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317881.480191:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317881.480194:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317881.480197:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317881.607640:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317881.735269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317882.631599:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317882.760210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317883.661390:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317883.783227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317884.680136:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317884.807239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317885.703419:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317885.831235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317886.605964:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317886.605975:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317886.605978:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317886.605982:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317886.727382:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317886.855218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317887.751390:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317887.879235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317888.775840:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317888.903230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317889.799280:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317889.930112:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317890.828543:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317890.951226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317891.719529:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317891.719540:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317891.719543:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317891.719546:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317891.847568:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317891.975212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317892.871709:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317892.999257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317893.895329:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317894.023186:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317894.920799:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317895.047205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317895.943311:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317896.074619:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317896.839448:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317896.839470:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317896.839473:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317896.839476:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317896.969239:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317897.095215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317897.992075:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317898.119310:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317899.016406:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317899.143280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317900.039283:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317900.167253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317901.063358:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317901.191209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317901.959232:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317876/real 1767317876] req@ffff88c5c57d5880 x1853163991631872/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317901 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317901.959267:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5880 x1853163991631872/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317901 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317901.959278:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317876/real 1767317876] req@ffff88c5c57d4380 x1853163991632000/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317901 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317901.959290:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4380 x1853163991632000/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317901 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317901.959327:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317901.960719:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317901.960728:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991631872:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317901.960737:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317901.960923:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317901.960928:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991632000:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317901.961024:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317901.961029:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317901.961032:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317901.961037:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3075 00000100:00080000:1.0:1767317901.961046:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317901.961052:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317901.961078:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cc000 x1853163991632128/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317901.961111:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317901.961114:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317901.961116:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317901.961120:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3075 00000100:00080000:1.0:1767317901.961123:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1767317901.961125:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991632128:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767317901.961127:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317901.961144:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317901.961919:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf800 x1853163991632256/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317901.962028:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991632256:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317902.087470:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317902.216144:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317903.111692:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317903.239840:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317904.135450:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317904.263406:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317905.159322:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317905.287263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317906.183628:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317906.311271:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317907.082784:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317907.082797:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317907.082801:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317907.082804:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317907.207469:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317907.335289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317908.231836:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317908.361051:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317909.255878:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317909.383332:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317910.282799:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317910.407238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317911.303320:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317911.431294:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317912.199334:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317912.199341:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317912.199344:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317912.199346:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317912.327846:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317912.455210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317913.351427:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317913.479393:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317914.375593:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317914.503256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317915.399314:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317915.527636:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317916.423361:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317916.551235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317917.322369:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317917.322399:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317917.322403:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317917.322406:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317917.447360:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317917.575203:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317918.471528:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317918.599312:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317919.497809:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317919.623584:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317920.519311:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317920.647267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317921.543540:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317921.671561:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317922.439423:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317922.439435:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317922.439439:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317922.439442:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317922.567471:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317922.695334:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317923.591371:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317923.719222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317924.615317:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317924.743215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317925.639358:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317925.767228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317926.535229:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317901/real 1767317901] req@ffff88c5c65cf800 x1853163991632256/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317926 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317926.535263:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf800 x1853163991632256/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317926 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317926.535274:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317901/real 1767317901] req@ffff88c5c65cc000 x1853163991632128/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317926 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317926.535286:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cc000 x1853163991632128/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317926 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317926.535316:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317926.535346:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317926.535352:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991632256:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767317926.535358:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317926.535383:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317926.535387:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991632128:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767317926.535520:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317926.535526:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767317926.535529:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317926.535541:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3101 00000100:00080000:2.0:1767317926.535550:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317926.535555:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767317926.535582:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5c00 x1853163991632384/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767317926.535603:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317926.535607:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767317926.535608:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317926.535612:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3101 00000100:00080000:2.0:1767317926.535615:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317926.535618:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767317926.535627:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767317926.535638:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4a80 x1853163991632512/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317926.535706:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991632384:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317926.535805:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991632512:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317926.663701:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317926.795164:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317927.687416:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317927.815243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317928.711411:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317928.839227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317929.736299:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317929.863926:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317930.759417:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317930.887227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317931.655248:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317931.655258:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317931.655261:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317931.655264:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317931.783296:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317931.911253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317932.807636:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317932.935230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317933.831381:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317933.959233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317934.855416:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317934.984030:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317935.879497:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317936.007307:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317936.775304:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317936.775319:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317936.775323:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317936.775354:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317936.907579:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317937.032599:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317937.927898:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317938.058676:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317938.951436:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317939.079225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317939.975556:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317940.103186:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317941.000563:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317941.129526:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317941.895328:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317941.895337:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317941.895340:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317941.895344:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317942.023319:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317942.151281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317943.047324:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317943.175223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317944.072395:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317944.199227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317945.095350:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317945.223318:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317946.121491:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317946.248393:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317947.015510:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317947.015521:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317947.015524:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317947.015527:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317947.144688:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317947.271242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317948.167320:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317948.295220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317949.191655:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317949.319198:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317950.215363:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317950.343224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767317951.114150:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317926/real 1767317926] req@ffff88c5c57d5c00 x1853163991632384/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317951 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317951.114183:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5c00 x1853163991632384/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317951 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317951.114194:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317926/real 1767317926] req@ffff88c5c57d4a80 x1853163991632512/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317951 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317951.114205:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4a80 x1853163991632512/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317951 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317951.114225:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317951.114245:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317951.114251:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991632384:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317951.114257:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317951.114264:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317951.114268:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991632512:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767317951.114284:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317951.114291:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767317951.114295:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317951.114301:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3125 00000100:00080000:2.0:1767317951.114310:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317951.114334:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767317951.114371:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7480 x1853163991632640/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767317951.114398:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317951.114401:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767317951.114403:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767317951.114406:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3125 00000100:00080000:2.0:1767317951.114410:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767317951.114414:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767317951.114424:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767317951.114435:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5f80 x1853163991632768/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317951.114508:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991632640:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317951.114598:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991632768:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317951.240197:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317951.367178:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317952.263353:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317952.392162:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317953.287280:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317953.415290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317954.311332:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317954.439231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317955.335344:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317955.463631:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767317956.231303:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317956.231313:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767317956.231317:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767317956.231321:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317956.359667:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317956.487266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317957.384270:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317957.511203:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317958.408373:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317958.537898:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317959.432423:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317959.559246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317960.455959:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317960.583739:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317961.351267:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317961.351278:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317961.351281:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317961.351284:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317961.479264:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317961.607231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317962.503381:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317962.631220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317963.530705:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317963.655383:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317964.551313:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317964.679261:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317965.575744:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317965.703277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767317966.471370:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317966.471384:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767317966.471387:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767317966.471391:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317966.599385:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317966.727699:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317967.623447:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317967.753772:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317968.647763:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317968.778361:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317969.671417:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317969.799321:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317970.695346:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317970.823236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317971.591501:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317971.591511:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317971.591514:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317971.591517:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317971.720584:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317971.852908:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317972.743784:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317972.871259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317973.767702:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317973.895280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317974.791265:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317974.919209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317975.815297:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317975.943630:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317976.711215:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317976.711225:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317976.711228:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317976.711231:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767317976.713037:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317951/real 1767317951] req@ffff88c5c57d7480 x1853163991632640/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317976 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317976.713070:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7480 x1853163991632640/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767317976 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317976.713082:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317951/real 1767317951] req@ffff88c5c57d5f80 x1853163991632768/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317976 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767317976.713125:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5f80 x1853163991632768/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767317976 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767317976.713145:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317976.713159:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317976.713163:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991632640:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767317976.713169:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767317976.713176:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767317976.713179:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991632768:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767317976.713301:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317976.713307:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767317976.713310:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317976.713315:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3150 00000100:00080000:1.0:1767317976.713324:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317976.713330:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767317976.713375:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cdf80 x1853163991632896/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767317976.713399:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317976.713403:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767317976.713405:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767317976.713408:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3150 00000100:00080000:1.0:1767317976.713412:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767317976.713416:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767317976.713424:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767317976.713435:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ced80 x1853163991633024/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767317976.713502:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991632896:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767317976.713597:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991633024:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767317976.839410:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317976.967232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317977.863433:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317977.991251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317978.887682:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317979.015511:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317979.911579:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317980.039229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317980.935838:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317981.063250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767317981.831404:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317981.831415:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767317981.831418:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767317981.831421:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317981.959424:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317982.087237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317982.983561:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317983.112137:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317984.007363:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317984.135320:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317985.033952:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317985.159267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317986.055367:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317986.184376:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317986.951525:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317986.951552:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317986.951555:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317986.951558:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317987.086268:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317987.208752:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317988.103973:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317988.231254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317989.127325:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317989.256148:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317990.151350:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317990.279972:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317991.175390:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317991.304663:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317992.071398:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317992.071408:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317992.071411:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317992.071414:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767317992.199384:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317992.327926:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317993.226596:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317993.351237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317994.247526:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317994.375279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317995.273782:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317995.401523:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767317996.295396:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767317996.426479:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767317997.191301:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317997.191313:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767317997.191316:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767317997.191319:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767317997.319338:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317997.448924:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317998.343272:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317998.473212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767317999.367474:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767317999.496390:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318000.393187:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318000.519550:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767318001.287215:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317976/real 1767317976] req@ffff88c5c65cdf80 x1853163991632896/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318001 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318001.287249:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cdf80 x1853163991632896/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318001 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318001.287260:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767317976/real 1767317976] req@ffff88c5c65ced80 x1853163991633024/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318001 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318001.287295:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ced80 x1853163991633024/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318001 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318001.287313:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318001.287340:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318001.287345:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991632896:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318001.287351:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318001.287357:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318001.287360:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991633024:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767318001.287503:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318001.287508:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767318001.287511:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318001.287516:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3175 00000100:00080000:1.0:1767318001.287525:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318001.287531:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767318001.287562:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cca80 x1853163991633152/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767318001.287598:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318001.287602:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767318001.287611:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318001.287615:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3175 00000100:00080000:1.0:1767318001.287619:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318001.287623:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767318001.287633:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767318001.287644:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cc380 x1853163991633280/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767318001.287762:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991633152:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767318001.287856:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991633280:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318001.415309:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318001.544827:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318002.439628:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318002.567275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318003.463402:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318003.591260:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318004.487486:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318004.615770:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318005.511331:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318005.639232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318006.408467:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318006.408478:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318006.408480:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318006.408483:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318006.535452:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318006.663239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318007.564389:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318007.687519:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318008.585672:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318008.712383:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318009.608887:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318009.735206:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318010.631393:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318010.759246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318011.527469:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318011.527477:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318011.527480:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318011.527482:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318011.655573:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318011.783222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318012.680296:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318012.807227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318013.703395:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318013.831283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318014.727473:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318014.855315:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318015.756810:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318015.879226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318016.647222:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318016.647233:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318016.647237:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318016.647241:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:0.0:1767318016.775307:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318016.903283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318017.799835:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318017.928415:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318018.824512:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318018.951233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318019.847810:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318019.975197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318020.871254:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318020.999296:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318021.767858:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318021.767866:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318021.767868:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318021.767871:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318021.895444:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318022.023210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318022.919326:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318023.047213:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318023.943798:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318024.071288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318024.968161:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318025.095453:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318025.991294:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318026.121288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318026.889175:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318026.889187:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318026.889190:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318026.889194:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767318026.889203:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318001/real 1767318001] req@ffff88c5c65cca80 x1853163991633152/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318026 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318026.889237:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cca80 x1853163991633152/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318026 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318026.889248:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318001/real 1767318001] req@ffff88c5c65cc380 x1853163991633280/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318026 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318026.889259:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cc380 x1853163991633280/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318026 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318026.889277:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318026.889353:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318026.889358:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991633152:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318026.889382:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318026.889587:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318026.889592:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318026.889601:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318026.889606:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3200 00000100:00080000:3.0:1767318026.889615:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318026.889622:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318026.889651:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728a80 x1853163991633408/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318026.889666:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318026.889670:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318026.889672:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318026.889676:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3200 00000100:00080000:3.0:1767318026.889679:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318026.889682:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318026.889692:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318026.889703:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b480 x1853163991633536/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767318026.889787:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318026.889791:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991633280:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767318026.889819:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991633408:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318026.889980:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991633536:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318027.017917:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318027.144758:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318028.040932:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318028.167411:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318029.063458:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318029.191194:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318030.088435:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318030.215799:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318031.111431:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318031.239305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318032.007278:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318032.007336:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318032.007339:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318032.007342:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318032.135432:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318032.263431:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318033.159253:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318033.287217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318034.186368:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318034.311255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318035.215633:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318035.335208:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318036.235386:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318036.359742:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318037.130283:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318037.130293:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318037.130295:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318037.130299:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:0.0:1767318037.255238:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318037.384126:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318038.279839:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318038.407235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318039.303318:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318039.431497:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318040.327669:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318040.455637:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318041.353764:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318041.479624:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318042.247259:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318042.247268:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318042.247272:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318042.247275:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318042.375647:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318042.503252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318043.400087:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318043.530055:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318044.426399:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318044.551220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318045.447527:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318045.575515:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318046.509004:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318046.599228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318047.367256:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318047.367269:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318047.367273:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318047.367276:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318047.559484:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318047.623678:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318048.583939:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318048.650188:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318049.611626:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318049.671216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318050.633695:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318050.695235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767318051.463227:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318026/real 1767318026] req@ffff88c5c5728a80 x1853163991633408/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318051 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318051.463263:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728a80 x1853163991633408/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318051 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318051.463274:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318026/real 1767318026] req@ffff88c5c572b480 x1853163991633536/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318051 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318051.463286:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b480 x1853163991633536/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318051 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318051.463303:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318051.463327:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318051.463332:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991633408:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318051.463338:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318051.463343:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318051.463346:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991633536:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767318051.464160:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318051.464167:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767318051.464171:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318051.464178:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3226 00000100:00080000:0.0:1767318051.464187:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318051.464193:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767318051.464227:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402300 x1853163991633664/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767318051.464254:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318051.464258:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767318051.464260:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318051.464264:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3226 00000100:00080000:0.0:1767318051.464267:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318051.464271:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767318051.464280:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767318051.464292:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402a00 x1853163991633792/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767318051.464365:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991633664:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767318051.464472:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991633792:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318051.657349:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318051.719348:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318052.679410:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318052.743493:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318053.705988:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318053.769042:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318054.727634:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318054.791353:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318055.757464:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318055.815192:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318056.588771:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318056.588783:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318056.588786:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318056.588790:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318056.778256:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318056.839793:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318057.799277:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318057.863226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318058.824649:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318058.887244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318059.848092:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318059.912347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318060.871355:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318060.935466:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318061.704072:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318061.704084:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318061.704087:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318061.704091:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318061.895599:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318061.959236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318062.920778:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318062.983215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318063.943599:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318064.007378:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318064.967981:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318065.031275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318065.991792:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318066.055257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318066.823308:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318066.823318:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318066.823321:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318066.823324:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318067.015822:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318067.079301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318068.039278:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318068.103276:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318069.065073:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318069.130148:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318070.087259:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318070.151842:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318071.115945:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318071.175211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318071.943914:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318071.943926:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318071.943929:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318071.943933:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318072.135349:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318072.199398:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318073.159732:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318073.232661:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318074.183735:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318074.247686:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318075.207343:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318075.271197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318076.039230:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318051/real 1767318051] req@ffff88c5c6402300 x1853163991633664/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318076 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318076.039263:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402300 x1853163991633664/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318076 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318076.039274:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318051/real 1767318051] req@ffff88c5c6402a00 x1853163991633792/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318076 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318076.039309:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402a00 x1853163991633792/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318076 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318076.039326:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318076.039348:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318076.039352:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991633664:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318076.039358:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318076.039364:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318076.039367:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991633792:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318076.043058:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318076.043069:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318076.043072:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318076.043082:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3250 00000100:00080000:2.0:1767318076.043092:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318076.043113:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318076.043145:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4000 x1853163991633920/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767318076.043171:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318076.043174:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318076.043176:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318076.043180:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3250 00000100:00080000:2.0:1767318076.043183:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318076.043187:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318076.043195:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767318076.043206:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6a00 x1853163991634048/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318076.043689:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991633920:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318076.044784:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991634048:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318076.231330:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318076.295349:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318077.255835:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318077.319198:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318078.280121:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318078.343218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318079.303671:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318079.367212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318080.327386:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318080.392289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318081.159515:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318081.159524:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318081.159527:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318081.159530:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318081.351250:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318081.415393:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318082.376687:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318082.439636:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318083.400115:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318083.463236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318084.423435:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318084.487223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318085.447685:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318085.511685:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318086.279607:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318086.279615:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318086.279617:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318086.279619:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318086.471244:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318086.535414:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318087.499734:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318087.559221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318088.519353:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318088.583232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318089.543407:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318089.607368:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318090.567336:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318090.631347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318091.399501:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318091.399513:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318091.399516:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318091.399520:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318091.591531:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318091.655223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318092.615411:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318092.679303:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318093.639400:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318093.703182:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318094.663276:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318094.728368:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318095.687404:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318095.755397:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318096.519828:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318096.519838:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318096.519841:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318096.519844:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318096.711561:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318096.775216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318097.736326:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318097.799228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318098.759492:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318098.823216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318099.783546:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318099.847278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318100.807455:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318100.871218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318101.639477:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1767318101.639488:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318076/real 1767318076] req@ffff88c5c57d4000 x1853163991633920/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318101 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318101.639488:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318101.639493:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318101.639502:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767318101.639527:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4000 x1853163991633920/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318101 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318101.639538:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318076/real 1767318076] req@ffff88c5c57d6a00 x1853163991634048/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318101 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318101.639588:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6a00 x1853163991634048/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318101 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318101.639607:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318101.639623:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318101.639628:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991633920:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318101.639632:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318101.639637:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318101.639640:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318101.639646:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318101.639647:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3275 00000100:00080000:3.0:1767318101.639661:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318101.639668:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00080000:3.0:1767318101.639669:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767318101.639676:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991634048:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767318101.639705:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b480 x1853163991634176/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318101.639734:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318101.639736:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318101.639738:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318101.639741:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3275 00000100:00080000:3.0:1767318101.639744:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318101.639747:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318101.639760:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318101.639769:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728a80 x1853163991634304/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318101.639843:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991634176:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318101.639934:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991634304:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318101.833835:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318101.897452:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318102.855245:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318102.919272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318103.879275:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318103.943932:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318104.903474:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318104.967427:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318105.927359:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318105.992541:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318106.759235:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318106.759246:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318106.759249:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318106.759252:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318106.951370:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318107.015268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318107.975434:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318108.040283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318108.999284:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318109.064269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318110.023277:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318110.087262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318111.048941:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318111.111229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318111.879391:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318111.879403:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318111.879406:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318111.879409:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318112.072282:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318112.135352:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318113.095522:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318113.159235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318114.119967:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318114.185303:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318115.144828:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318115.208428:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318116.168781:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318116.231361:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318117.002288:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318117.002307:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318117.002311:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318117.002321:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318117.191760:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318117.255404:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318118.215276:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318118.279394:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318119.239749:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318119.304820:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318120.268441:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318120.327288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318121.289874:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318121.351355:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318122.119352:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318122.119362:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318122.119365:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318122.119369:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318122.311407:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318122.375311:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318123.335322:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318123.399687:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318124.359865:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318124.423224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318125.383618:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318125.447337:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318126.215666:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318101/real 1767318101] req@ffff88c5c572b480 x1853163991634176/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318126 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318126.215700:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b480 x1853163991634176/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318126 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318126.215711:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318101/real 1767318101] req@ffff88c5c5728a80 x1853163991634304/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318126 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318126.215723:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728a80 x1853163991634304/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318126 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318126.215741:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318126.215767:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318126.215771:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991634176:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318126.215778:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318126.215783:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318126.215804:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991634304:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767318126.215879:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318126.215886:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767318126.215889:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318126.215895:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3300 00000100:00080000:0.0:1767318126.215903:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318126.215909:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767318126.215938:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402a00 x1853163991634432/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767318126.215967:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318126.215978:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767318126.215980:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318126.215985:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3300 00000100:00080000:0.0:1767318126.215989:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318126.215993:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767318126.216002:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767318126.216015:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402300 x1853163991634560/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318126.216090:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991634432:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318126.216248:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991634560:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318126.407503:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318126.471229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318127.431539:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318127.495227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318128.455540:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318128.519212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318129.481295:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318129.543236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318130.503357:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318130.567303:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318131.335408:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318131.335419:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318131.335422:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318131.335426:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318131.529823:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318131.591237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318132.551754:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318132.615218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318133.575616:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318133.639262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318134.601385:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318134.663230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318135.623945:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318135.687209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318136.455215:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318136.455225:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318136.455228:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318136.455231:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318136.647315:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318136.711211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318137.671921:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318137.736072:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318138.695359:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318138.759240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318139.719613:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318139.786257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318140.746396:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318140.807278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318141.575314:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318141.575326:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318141.575329:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318141.575333:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318141.768583:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318141.831196:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318142.792287:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318142.855199:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318143.815572:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318143.879337:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318144.839395:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318144.903324:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318145.863323:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318145.927260:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318146.695383:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318146.695395:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318146.695398:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318146.695401:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318146.891135:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318146.951314:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318147.911649:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318147.975241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318148.936161:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318148.999303:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318149.959720:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318150.023225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318150.986136:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318151.047221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318151.815459:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318151.815469:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318151.815472:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318151.815476:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767318151.815573:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318126/real 1767318126] req@ffff88c5c6402a00 x1853163991634432/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318151 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318151.815607:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402a00 x1853163991634432/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318151 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318151.815617:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318126/real 1767318126] req@ffff88c5c6402300 x1853163991634560/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318151 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318151.815628:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402300 x1853163991634560/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318151 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318151.815646:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318151.815667:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318151.815671:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991634432:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318151.815676:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318151.815680:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318151.815683:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991634560:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767318151.815743:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318151.815748:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767318151.815751:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318151.815756:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3325 00000100:00080000:0.0:1767318151.815765:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318151.815771:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767318151.815815:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402300 x1853163991634688/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767318151.815841:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318151.815844:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767318151.815845:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318151.815849:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3325 00000100:00080000:0.0:1767318151.815853:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318151.815857:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767318151.815866:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767318151.815877:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402a00 x1853163991634816/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318151.815937:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991634688:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318151.816033:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991634816:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318152.007279:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318152.071205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318153.031289:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318153.097290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318154.055364:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318154.119239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318155.079759:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318155.143219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318156.103268:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318156.167964:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318156.935224:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318156.935236:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318156.935239:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318156.935242:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318157.127235:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318157.191482:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318158.151318:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318158.216169:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318159.176667:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318159.239439:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318160.199848:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318160.264959:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318161.223573:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318161.289128:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318162.055259:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318162.055270:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318162.055273:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318162.055293:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318162.248665:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318162.311409:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318163.271445:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318163.336029:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318164.295373:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318164.361244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318165.320726:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318165.383217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318166.343320:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318166.407229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318167.176983:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318167.176993:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318167.176996:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318167.177000:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318167.367313:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318167.431224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318168.394126:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318168.455258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318169.415218:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318169.479220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318170.439313:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318170.503302:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318171.463421:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318171.527548:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318172.295477:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318172.295486:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318172.295489:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318172.295493:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318172.487989:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318172.551272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318173.513502:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318173.575430:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318174.535338:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318174.599309:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318175.559689:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318175.623510:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318176.391888:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318151/real 1767318151] req@ffff88c5c6402300 x1853163991634688/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318176 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318176.391921:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402300 x1853163991634688/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318176 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318176.391932:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318151/real 1767318151] req@ffff88c5c6402a00 x1853163991634816/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318176 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318176.391943:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402a00 x1853163991634816/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318176 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318176.391960:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318176.391985:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318176.391989:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991634688:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318176.391995:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318176.392000:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318176.392003:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991634816:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318176.392351:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318176.392357:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318176.392360:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318176.392365:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3351 00000100:00080000:2.0:1767318176.392374:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318176.392381:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318176.392417:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7800 x1853163991634944/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767318176.392443:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318176.392446:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318176.392448:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318176.392452:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3351 00000100:00080000:2.0:1767318176.392455:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318176.392459:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318176.392468:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767318176.392479:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6300 x1853163991635072/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318176.392894:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991634944:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318176.393009:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991635072:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318176.583661:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318176.647744:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318177.607368:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318177.671243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318178.631384:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318178.696768:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318179.655635:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318179.719215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318180.679633:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318180.743528:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318181.512410:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318181.512421:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318181.512424:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318181.512428:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318181.703454:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318181.769682:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318182.727309:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318182.791616:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318183.751496:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318183.815241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318184.775969:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318184.839200:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318185.799327:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318185.863274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318186.631784:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318186.631794:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318186.631797:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318186.631801:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318186.823601:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318186.887228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318187.847448:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318187.911360:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318188.873269:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318188.935719:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318189.896988:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318189.959305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318190.920007:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318190.983347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318191.752156:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318191.752166:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318191.752169:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318191.752172:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318191.943353:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318192.007211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318192.967372:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318193.032166:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318193.991356:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318194.055226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318195.015394:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318195.079251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318196.039969:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318196.103247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318196.871324:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318196.871335:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318196.871338:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318196.871342:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318197.063383:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318197.127756:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318198.087318:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318198.151299:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318199.111658:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318199.175284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318200.137370:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318200.199632:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318201.159568:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318201.223227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318201.991246:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318176/real 1767318176] req@ffff88c5c57d7800 x1853163991634944/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318201 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318201.991302:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7800 x1853163991634944/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318201 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318201.991313:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318176/real 1767318176] req@ffff88c5c57d6300 x1853163991635072/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318201 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318201.991325:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6300 x1853163991635072/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318201 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318201.991343:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318201.991365:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318201.991369:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991634944:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318201.991375:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318201.991380:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318201.991383:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991635072:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767318201.992961:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318201.992969:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318201.992972:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318201.992979:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3375 00000100:00080000:3.0:1767318201.992988:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318201.993041:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318201.993078:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728e00 x1853163991635200/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318201.993135:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318201.993140:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318201.993142:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318201.993146:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3375 00000100:00080000:3.0:1767318201.993150:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318201.993153:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318201.993160:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318201.993169:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729880 x1853163991635328/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318201.993321:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991635200:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318201.993456:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991635328:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318202.183462:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318202.248918:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318203.210992:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318203.271271:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318204.231979:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318204.295243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318205.255556:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318205.321185:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318206.282121:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318206.343287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318207.113600:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318207.113613:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318207.113616:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318207.113619:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318207.303390:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318207.370452:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318208.329366:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318208.391728:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318209.351292:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318209.416390:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318210.376819:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318210.439263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318211.399360:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318211.463280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318212.233016:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318212.233026:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318212.233029:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318212.233032:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318212.424507:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318212.487222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318213.447325:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318213.511217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318214.473434:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318214.535248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318215.495890:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318215.559297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318216.519429:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318216.583221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318217.352277:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318217.352301:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318217.352305:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318217.352308:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318217.543334:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318217.607288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318218.567950:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318218.632982:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318219.591397:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318219.655231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318220.615335:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318220.679217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318221.639277:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318221.703200:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318222.471214:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318222.471224:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318222.471227:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318222.471231:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318222.663315:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318222.727223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318223.689059:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318223.751256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318224.711710:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318224.775216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318225.735802:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318225.799224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318226.567250:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318201/real 1767318201] req@ffff88c5c5728e00 x1853163991635200/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318226 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318226.567284:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728e00 x1853163991635200/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318226 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318226.567295:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318201/real 1767318201] req@ffff88c5c5729880 x1853163991635328/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318226 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318226.567333:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729880 x1853163991635328/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318226 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318226.567352:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318226.567380:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318226.567385:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991635200:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318226.567391:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318226.567396:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318226.567400:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991635328:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318226.568170:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318226.568178:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318226.568181:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318226.568188:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3401 00000100:00080000:2.0:1767318226.568196:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318226.568203:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318226.568232:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5500 x1853163991635456/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767318226.568259:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318226.568263:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318226.568265:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318226.568268:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3401 00000100:00080000:2.0:1767318226.568272:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318226.568288:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318226.568296:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767318226.568308:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4e00 x1853163991635584/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318226.570378:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991635456:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318226.570489:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991635584:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318226.759675:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318226.823768:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318227.783624:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318227.847329:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318228.807324:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318228.871302:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318229.833282:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318229.895398:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318230.855331:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318230.919249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318231.687226:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318231.687236:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318231.687239:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318231.687242:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318231.879314:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318231.945427:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318232.903660:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318232.967203:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318233.927394:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318233.991217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318234.952020:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318235.015512:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318235.975355:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318236.041534:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318236.807255:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318236.807267:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318236.807270:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318236.807273:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318236.999871:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318237.063217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318238.024481:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318238.089591:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318239.047335:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318239.111226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318240.074952:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318240.135223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318241.095653:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318241.162051:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318241.928318:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318241.928361:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318241.928364:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318241.928368:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318242.121910:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318242.183284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318243.143382:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318243.207243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318244.168188:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318244.231231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318245.191464:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318245.255258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318246.215853:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318246.279340:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318247.047243:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318247.047253:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318247.047256:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318247.047260:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318247.241310:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318247.303666:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318248.265134:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318248.327243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318249.287312:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318249.351736:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318250.311605:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318250.375257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318251.143343:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318226/real 1767318226] req@ffff88c5c57d5500 x1853163991635456/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318251 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318251.143377:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5500 x1853163991635456/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318251 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318251.143387:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318226/real 1767318226] req@ffff88c5c57d4e00 x1853163991635584/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318251 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318251.143398:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4e00 x1853163991635584/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318251 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318251.143416:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318251.143443:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318251.143447:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991635456:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318251.143453:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318251.143458:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318251.143479:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991635584:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767318251.144451:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318251.144478:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767318251.144481:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318251.144487:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3425 00000100:00080000:0.0:1767318251.144497:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318251.144504:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767318251.144534:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401f80 x1853163991635712/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767318251.144570:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318251.144574:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767318251.144576:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318251.144580:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3425 00000100:00080000:0.0:1767318251.144584:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318251.144587:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767318251.144597:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767318251.144608:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403480 x1853163991635840/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318251.144742:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991635712:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318251.144844:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991635840:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318251.335487:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318251.399639:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318252.359294:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318252.423274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318253.383438:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318253.447244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318254.407323:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318254.471218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318255.431647:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318255.495227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318256.263278:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318256.263288:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318256.263291:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318256.263313:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318256.456891:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318256.519231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318257.479315:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318257.543197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318258.503610:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318258.567258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318259.527549:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318259.591235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318260.551561:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318260.615698:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318261.383598:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318261.383608:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318261.383610:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318261.383614:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318261.576273:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318261.640549:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318262.600689:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318262.663297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318263.623782:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318263.687197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318264.647535:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318264.712924:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318265.671350:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318265.735315:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318266.503433:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318266.503444:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318266.503447:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318266.503450:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318266.695833:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318266.759651:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318267.720726:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318267.783249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318268.748069:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318268.807785:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318269.767378:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318269.832131:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318270.791421:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318270.855803:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318271.626593:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318271.626604:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318271.626607:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318271.626611:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318271.816022:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318271.879237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318272.841953:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318272.903323:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318273.866062:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318273.928454:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318274.888797:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318274.951224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318275.912128:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318275.975542:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318276.743273:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318251/real 1767318251] req@ffff88c5c6401f80 x1853163991635712/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318276 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318276.743311:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401f80 x1853163991635712/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318276 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318276.743342:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318251/real 1767318251] req@ffff88c5c6403480 x1853163991635840/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318276 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318276.743357:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403480 x1853163991635840/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318276 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318276.743376:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318276.743386:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318276.743390:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991635712:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318276.743395:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318276.743399:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318276.743402:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991635840:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767318276.743617:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318276.743623:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767318276.743626:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318276.743632:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3450 00000100:00080000:1.0:1767318276.743641:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318276.743648:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767318276.743694:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cce00 x1853163991635968/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767318276.743719:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318276.743722:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767318276.743724:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318276.743728:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3450 00000100:00080000:1.0:1767318276.743732:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318276.743736:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767318276.743746:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767318276.743759:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf480 x1853163991636096/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318276.744423:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991635968:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318276.744518:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991636096:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318276.935479:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318276.999229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318277.959326:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318278.023258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318278.983335:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318279.047547:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318280.008982:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318280.071803:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318281.032067:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318281.095222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318281.866069:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318281.866080:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318281.866083:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318281.866086:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318282.055361:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318282.119205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318283.079330:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318283.143425:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318284.103993:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318284.167232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318285.127261:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318285.191220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318286.151555:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318286.215591:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318286.983440:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318286.983450:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318286.983453:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318286.983456:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318287.175535:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318287.239408:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318288.199417:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318288.263247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318289.223632:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318289.287276:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318290.248552:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318290.311251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318291.271293:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318291.335204:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318292.105466:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318292.105477:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318292.105480:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318292.105484:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318292.296617:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318292.360034:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318293.319858:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318293.383898:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318294.345900:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318294.408043:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318295.367362:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318295.431287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318296.393500:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318296.455995:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318297.227184:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318297.227195:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318297.227198:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318297.227201:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318297.415502:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318297.480452:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318298.440718:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318298.503332:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318299.465918:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318299.528768:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318300.491376:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318300.551225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318301.319407:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318276/real 1767318276] req@ffff88c5c65cce00 x1853163991635968/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318301 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318301.319433:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cce00 x1853163991635968/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318301 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318301.319440:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318276/real 1767318276] req@ffff88c5c65cf480 x1853163991636096/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318301 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318301.319448:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf480 x1853163991636096/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318301 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318301.319480:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318301.320087:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318301.320118:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991635968:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318301.320127:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318301.320134:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318301.320137:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991636096:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318301.320405:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318301.320411:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318301.320415:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318301.320420:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3475 00000100:00080000:2.0:1767318301.320430:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318301.320439:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318301.320473:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4e00 x1853163991636224/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318301.321405:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991636224:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767318301.321587:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318301.321591:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318301.321594:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318301.321599:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3475 00000100:00080000:2.0:1767318301.321604:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318301.321609:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318301.321619:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767318301.321638:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5500 x1853163991636352/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318301.321796:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991636352:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318301.511490:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318301.575233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318302.535876:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318302.600285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318303.559570:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318303.623996:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318304.584066:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318304.647185:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318305.607374:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318305.671293:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318306.442287:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318306.442296:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318306.442299:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318306.442303:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318306.631374:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318306.695299:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318307.655646:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318307.719686:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318308.679315:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318308.744010:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318309.703323:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318309.767445:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318310.727292:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318310.791232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318311.559819:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318311.559851:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318311.559854:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318311.559857:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318311.751331:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318311.815249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318312.775557:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318312.839334:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318313.800849:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318313.863221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318314.823287:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318314.888334:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318315.847374:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318315.911311:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318316.679386:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318316.679397:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318316.679400:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318316.679403:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318316.872420:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318316.935234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318317.896270:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318317.959551:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318318.919323:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318318.983633:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318319.943428:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318320.007432:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318320.967299:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318321.032582:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318321.799450:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318321.799460:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318321.799463:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318321.799467:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318321.995630:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318322.055220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318323.015434:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318323.079532:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318324.039476:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318324.103243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318325.064654:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318325.132148:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318326.105141:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318326.153604:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318326.921825:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318326.921837:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318326.921841:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318326.921844:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767318326.923559:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318301/real 1767318301] req@ffff88c5c57d5500 x1853163991636352/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318326 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318326.923593:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5500 x1853163991636352/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318326 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318326.923604:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318301/real 1767318301] req@ffff88c5c57d4e00 x1853163991636224/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318326 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318326.923615:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4e00 x1853163991636224/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318326 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318326.923648:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318326.923676:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318326.923680:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991636352:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767318326.923687:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318326.923702:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318326.923705:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991636224:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318326.923787:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318326.923792:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767318326.923795:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318326.923800:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3500 00000100:00080000:1.0:1767318326.923809:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318326.923815:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767318326.923849:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf100 x1853163991636480/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767318326.923872:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318326.923876:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767318326.923877:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318326.923880:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3500 00000100:00080000:1.0:1767318326.923884:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318326.923888:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767318326.923903:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767318326.923914:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd880 x1853163991636608/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318326.924184:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991636480:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318326.924273:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991636608:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318327.111602:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318327.175201:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318328.135563:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318328.199447:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318329.159306:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318329.224009:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318330.183292:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318330.247233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318331.207509:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318331.271224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318332.039363:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318332.039375:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318332.039396:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318332.039400:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318332.231974:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318332.295285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318333.255394:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318333.319233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318334.281163:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318334.343620:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318335.303343:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318335.367413:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318336.327611:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318336.391172:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318337.162783:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318337.162793:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318337.162796:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318337.162799:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318337.356310:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318337.415182:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318338.375312:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318338.439252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318339.399405:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318339.463238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318340.423464:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318340.487245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318341.449799:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318341.513684:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318342.279276:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318342.279286:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318342.279289:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318342.279292:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318342.471414:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318342.536348:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318343.495594:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318343.561458:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318344.521293:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318344.583239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318345.543452:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318345.607231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318346.568662:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318346.631361:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318347.399236:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318347.399247:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318347.399250:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318347.399253:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318347.591704:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318347.655225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318348.616366:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318348.679304:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318349.639299:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318349.703196:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318350.663287:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318350.727217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318351.495272:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318326/real 1767318326] req@ffff88c5c65cf100 x1853163991636480/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318351 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318351.495312:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf100 x1853163991636480/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318351 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318351.495323:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318326/real 1767318326] req@ffff88c5c65cd880 x1853163991636608/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318351 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318351.495333:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd880 x1853163991636608/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318351 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318351.495351:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318351.495375:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318351.495379:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991636480:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318351.495385:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318351.495390:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318351.495393:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991636608:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767318351.495547:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318351.495553:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767318351.495556:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318351.495561:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3526 00000100:00080000:0.0:1767318351.495571:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318351.495576:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767318351.495608:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403480 x1853163991636736/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767318351.495632:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318351.495649:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767318351.495651:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318351.495655:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3526 00000100:00080000:0.0:1767318351.495659:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318351.495663:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767318351.495671:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767318351.495682:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401f80 x1853163991636864/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318351.495743:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991636736:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318351.495884:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991636864:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318351.689429:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318351.751240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318352.711549:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318352.775326:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318353.735404:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318353.799289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318354.759366:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318354.823227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318355.784883:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318355.847243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318356.619746:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318356.619755:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318356.619775:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318356.619778:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318356.807257:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318356.871254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318357.831510:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318357.896323:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318358.855894:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318358.919267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318359.879401:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318359.943605:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318360.903489:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318360.968279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318361.735905:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318361.735914:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318361.735917:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318361.735920:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318361.927914:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318361.991351:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318362.951435:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318363.015232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318363.977200:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318364.040430:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318365.000775:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318365.063274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318366.023514:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318366.087227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318366.856596:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318366.856608:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318366.856611:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318366.856692:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318367.047342:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318367.111266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318368.071854:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318368.136669:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318369.095496:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318369.159662:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318370.120407:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318370.183270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318371.143357:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318371.207489:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318371.975508:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318371.975524:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318371.975528:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318371.975532:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318372.167698:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318372.231365:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318373.192514:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318373.255293:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318374.215352:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318374.279278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318375.239372:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318375.303270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767318376.071365:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318351/real 1767318351] req@ffff88c5c6403480 x1853163991636736/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318376 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767318376.071403:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403480 x1853163991636736/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318376 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767318376.071411:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318351/real 1767318351] req@ffff88c5c6401f80 x1853163991636864/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318376 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767318376.071420:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401f80 x1853163991636864/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318376 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767318376.071437:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767318376.071467:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767318376.071470:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991636736:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767318376.071475:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767318376.071502:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767318376.071505:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991636864:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767318376.072054:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318376.072062:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767318376.072066:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318376.072072:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3550 00000100:00080000:1.0:1767318376.072085:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318376.072496:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767318376.072706:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400700 x1853163991636992/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767318376.072792:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991636992:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318376.073519:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318376.073526:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767318376.073530:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318376.073537:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3550 00000100:00080000:0.0:1767318376.073544:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318376.073559:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767318376.073571:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767318376.073592:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403800 x1853163991637120/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767318376.074047:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991637120:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318376.264135:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318376.327500:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318377.288236:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318377.352061:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318378.312992:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318378.375267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318379.335412:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318379.399429:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318380.359883:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318380.423248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318381.191506:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318381.191526:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318381.191530:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318381.191533:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318381.383440:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318381.447686:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318382.407753:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318382.471717:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318383.431518:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318383.495654:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318384.456193:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318384.519305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318385.479994:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318385.543944:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318386.311779:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318386.311790:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318386.311793:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318386.311795:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318386.503437:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318386.567226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318387.527307:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318387.591263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318388.551573:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318388.615223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318389.575353:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318389.639240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318390.599453:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318390.663217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318391.431335:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318391.431346:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318391.431366:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318391.431370:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318391.623372:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318391.687249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318392.647395:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318392.711217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318393.671341:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318393.735698:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318394.695567:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318394.759272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318395.719257:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318395.783243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318396.551782:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318396.551794:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318396.551796:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318396.551799:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318396.747340:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318396.807459:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318397.767643:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318397.831273:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318398.791493:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318398.855252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318399.815428:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318399.879594:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318400.840704:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318400.903222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318401.671227:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318401.671238:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318401.671243:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318401.671246:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767318401.671295:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318376/real 1767318376] req@ffff88c5c6403800 x1853163991637120/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318401 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318401.671379:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403800 x1853163991637120/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318401 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318401.671391:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318376/real 1767318376] req@ffff88c5c6400700 x1853163991636992/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318401 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318401.671770:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400700 x1853163991636992/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318401 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318401.671797:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318401.671825:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318401.671830:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991637120:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767318401.671837:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318401.671844:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318401.671847:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991636992:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318401.672777:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318401.672788:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318401.672792:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318401.672804:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3575 00000100:00080000:3.0:1767318401.672816:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318401.672824:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318401.672863:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728380 x1853163991637248/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318401.672897:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318401.672900:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318401.672902:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318401.672924:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3575 00000100:00080000:3.0:1767318401.672928:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318401.672932:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318401.672940:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318401.673067:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728700 x1853163991637376/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318401.673655:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991637248:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318401.673772:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991637376:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318401.863335:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318401.928197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318402.888897:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318402.952083:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318403.911586:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318403.975422:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318404.935419:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318404.999237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318405.963374:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318406.023229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318406.791221:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318406.791232:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318406.791235:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318406.791239:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318406.984347:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318407.047313:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318408.009199:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318408.071250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318409.034257:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318409.095348:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318410.056565:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318410.119225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318411.080577:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318411.146542:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318411.911285:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318411.911296:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318411.911299:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318411.911301:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318412.103413:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318412.167244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318413.127806:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318413.191351:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318414.152805:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318414.215319:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318415.175737:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318415.239923:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318416.200127:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318416.263396:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318417.031197:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318417.031206:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318417.031209:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318417.031212:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318417.223389:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318417.293278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318418.247315:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318418.311230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318419.271598:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318419.335252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318420.296678:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318420.359687:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318421.319335:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318421.383187:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318422.151226:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318422.151237:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318422.151246:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318422.151249:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318422.343896:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318422.407711:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318423.367326:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318423.431900:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318424.391285:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318424.455446:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318425.417427:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318425.480869:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318426.247946:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318401/real 1767318401] req@ffff88c5c5728380 x1853163991637248/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318426 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318426.247979:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728380 x1853163991637248/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318426 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318426.247990:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318401/real 1767318401] req@ffff88c5c5728700 x1853163991637376/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318426 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318426.248004:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728700 x1853163991637376/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318426 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318426.248022:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318426.248049:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318426.248053:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991637248:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318426.248059:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318426.248064:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318426.248067:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991637376:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767318426.249123:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318426.249140:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318426.249145:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318426.249151:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3600 00000100:00080000:3.0:1767318426.249161:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318426.249167:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318426.249200:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b100 x1853163991637504/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318426.249228:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318426.249231:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318426.249233:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318426.249237:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3600 00000100:00080000:3.0:1767318426.249255:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318426.249259:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318426.249267:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318426.249279:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729180 x1853163991637632/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318426.249397:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991637504:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318426.249510:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991637632:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318426.441043:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318426.503230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318427.465418:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318427.527511:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318428.487391:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318428.551304:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318429.512773:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318429.575444:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318430.535300:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318430.599186:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318431.367500:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318431.367511:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318431.367514:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318431.367517:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318431.560801:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318431.623225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318432.583422:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318432.647238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318433.607421:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318433.671239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318434.632018:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318434.695219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318435.655288:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318435.720171:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318436.488656:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318436.488665:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318436.488667:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318436.488670:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318436.679312:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318436.743349:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318437.703593:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318437.767264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318438.727203:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318438.791960:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318439.752996:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318439.815220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318440.775965:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318440.839525:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318441.611997:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318441.612010:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318441.612013:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318441.612016:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318441.799298:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318441.864931:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318442.823900:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318442.887953:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318443.847291:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318443.911605:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318444.871579:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318444.935221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318445.898138:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318445.959232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318446.735341:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318446.735350:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318446.735363:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318446.735366:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318446.919307:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318446.983306:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318447.943431:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318448.007250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318448.967467:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318449.031297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318449.991500:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318450.055248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318451.015370:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318451.079229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318451.847272:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318451.847282:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318451.847294:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318451.847297:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767318451.847461:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318426/real 1767318426] req@ffff88c5c572b100 x1853163991637504/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318451 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767318451.847513:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b100 x1853163991637504/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318451 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767318451.847524:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318426/real 1767318426] req@ffff88c5c5729180 x1853163991637632/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318451 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767318451.847534:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729180 x1853163991637632/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318451 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767318451.847551:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767318451.847605:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767318451.847609:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991637504:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767318451.847616:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318451.853235:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318451.853245:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767318451.853249:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318451.853259:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3625 00000100:00080000:1.0:1767318451.853269:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318451.853276:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767318451.858889:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd880 x1853163991637760/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767318451.858916:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318451.858921:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767318451.858923:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318451.858929:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3625 00000100:00080000:1.0:1767318451.858939:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318451.858962:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767318451.858974:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767318451.858989:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf100 x1853163991637888/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767318451.859210:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767318451.859217:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991637632:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767318451.859255:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991637760:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767318451.859364:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991637888:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767318452.044832:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318452.103230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318453.065811:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318453.127413:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318454.088462:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318454.151212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318455.112143:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318455.175581:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318456.136528:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318456.201525:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318456.967466:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318456.967482:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318456.967485:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318456.967489:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318457.159730:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318457.223374:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318458.183442:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318458.249322:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318459.216625:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318459.271521:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318460.231617:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318460.299606:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318461.255402:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318461.319701:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318462.087954:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318462.087983:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318462.087987:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318462.087990:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318462.279746:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318462.343524:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318463.304836:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318463.367329:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318464.327628:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318464.391341:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318465.351629:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318465.415296:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318466.378813:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318466.439284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318467.208783:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318467.208798:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318467.208802:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318467.208806:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318467.399824:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318467.463348:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318468.423481:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318468.488366:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318469.447550:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318469.512481:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318470.472812:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318470.536542:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318471.495904:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318471.561035:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318472.327497:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318472.327513:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318472.327516:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318472.327520:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318472.523241:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318472.583585:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318473.545510:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318473.607277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318474.569972:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318474.631242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318475.598200:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318475.655478:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318476.423236:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318451/real 1767318451] req@ffff88c5c65cd880 x1853163991637760/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318476 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318476.423272:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd880 x1853163991637760/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318476 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318476.423284:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318451/real 1767318451] req@ffff88c5c65cf100 x1853163991637888/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318476 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318476.423296:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf100 x1853163991637888/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318476 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318476.423317:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318476.423344:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318476.423349:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991637760:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318476.423355:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318476.423360:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318476.423363:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991637888:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767318476.423462:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318476.423467:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318476.423470:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318476.423476:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3651 00000100:00080000:3.0:1767318476.423485:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318476.423492:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318476.423526:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728000 x1853163991638016/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318476.423550:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318476.423553:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318476.423569:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318476.423574:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3651 00000100:00080000:3.0:1767318476.423578:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318476.423581:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318476.423590:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318476.423601:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572ad80 x1853163991638144/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318476.423848:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991638016:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318476.423949:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991638144:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767318476.615666:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318476.679198:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318477.639335:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318477.703481:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318478.663423:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318478.727404:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318479.687874:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318479.751241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318480.711876:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318480.776677:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318481.543374:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318481.543384:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318481.543386:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318481.543389:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318481.739796:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318481.799196:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318482.760992:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318482.823222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318483.783280:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318483.847245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318484.807348:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318484.872046:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318485.832884:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318485.895226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318486.663347:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318486.663367:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318486.663370:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318486.663373:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318486.859042:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318486.919457:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318487.879768:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318487.943221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318488.903354:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318488.967217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318489.927898:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318489.996349:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318490.951349:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318491.015420:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318491.783513:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318491.783539:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318491.783542:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318491.783545:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318491.975420:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318492.039464:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318492.999430:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318493.065396:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318494.023431:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318494.087306:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318495.052127:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318495.112860:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318496.071634:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318496.135290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318496.903531:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318496.903547:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318496.903551:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318496.903555:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318497.103779:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318497.159608:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318498.119423:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318498.184271:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318499.143877:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318499.207286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318500.167408:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318500.231250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318501.191273:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318501.255236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767318502.023246:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318476/real 1767318476] req@ffff88c5c5728000 x1853163991638016/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318501 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318502.023278:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1767318502.023282:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728000 x1853163991638016/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318501 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318502.023286:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318502.023291:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318502.023297:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767318502.023302:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318476/real 1767318476] req@ffff88c5c572ad80 x1853163991638144/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318501 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767318502.023315:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572ad80 x1853163991638144/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318501 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767318502.023344:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767318502.023365:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767318502.023369:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991638016:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767318502.023376:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318502.023399:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318502.023405:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318502.023408:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318502.023415:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3675 00000100:00080000:3.0:1767318502.023425:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318502.023429:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767318502.023434:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991638144:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767318502.023444:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318502.031163:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729f80 x1853163991638272/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318502.031203:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318502.031209:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318502.031212:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318502.031217:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3675 00000100:00080000:3.0:1767318502.031229:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318502.031236:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318502.031248:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318502.031263:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729c00 x1853163991638400/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767318502.031982:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991638272:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767318502.032127:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991638400:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318502.215348:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318502.279215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318503.239488:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318503.304519:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318504.263311:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318504.327330:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318505.287380:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318505.351234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318506.312191:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318506.375395:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318507.146165:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318507.146176:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318507.146179:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318507.146182:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318507.335426:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318507.401421:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318508.361894:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318508.423366:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318509.383371:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318509.447281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318510.407447:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318510.471276:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318511.431521:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318511.495361:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318512.269234:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318512.269245:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318512.269248:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318512.269252:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318512.461386:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318512.519407:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318513.480033:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318513.543350:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318514.505774:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318514.567236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318515.527812:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318515.591429:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318516.551458:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318516.615298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318517.385405:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318517.385417:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318517.385420:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318517.385424:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318517.576677:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318517.639222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318518.599309:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318518.664130:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318519.623317:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318519.687775:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318520.647497:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318520.711234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318521.671498:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318521.735228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318522.503334:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318522.503345:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318522.503348:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318522.503351:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318522.695930:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318522.759263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318523.719609:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318523.783833:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318524.743344:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318524.807250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318525.768915:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318525.831945:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318526.791915:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318526.855222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318527.623270:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318502/real 1767318502] req@ffff88c5c5729f80 x1853163991638272/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318527 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318527.623304:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729f80 x1853163991638272/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318527 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318527.623315:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318502/real 1767318502] req@ffff88c5c5729c00 x1853163991638400/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318527 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318527.623327:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729c00 x1853163991638400/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318527 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318527.623346:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318527.623367:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318527.623371:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991638272:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318527.623377:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318527.623383:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318527.623399:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991638400:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767318527.623518:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318527.623524:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767318527.623528:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318527.623535:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3701 00000100:00080000:0.0:1767318527.623544:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318527.623567:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767318527.623597:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401500 x1853163991638528/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767318527.623622:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318527.623626:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767318527.623628:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318527.623632:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3701 00000100:00080000:0.0:1767318527.623636:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318527.623640:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767318527.623649:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767318527.623671:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401180 x1853163991638656/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318527.623800:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991638528:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318527.623910:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991638656:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767318527.815663:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318527.879237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318528.840013:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318528.903217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318529.865630:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318529.927317:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318530.889612:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318530.951323:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318531.913195:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318531.975661:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318532.743448:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318532.743477:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318532.743480:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318532.743483:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318532.936247:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318532.999788:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318533.966071:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318534.024175:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318534.983434:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318535.047341:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318536.008264:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318536.071796:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318537.031429:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318537.095421:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318537.863240:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318537.863251:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318537.863254:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318537.863257:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318538.060589:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318538.119397:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318539.079280:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318539.143230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318540.103423:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318540.167266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318541.127393:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318541.191223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318542.152065:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318542.216128:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318542.984231:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318542.984241:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318542.984244:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318542.984263:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318543.175458:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318543.239224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318544.199756:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318544.263229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318545.223327:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318545.287220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318546.248178:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318546.311211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318547.271503:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318547.335952:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318548.104366:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318548.104376:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318548.104379:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318548.104383:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318548.295628:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318548.359287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318549.319325:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318549.383212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318550.343731:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318550.411528:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318551.374186:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318551.431680:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318552.199344:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318527/real 1767318527] req@ffff88c5c6401500 x1853163991638528/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318552 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318552.199390:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401500 x1853163991638528/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318552 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318552.199403:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318527/real 1767318527] req@ffff88c5c6401180 x1853163991638656/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318552 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318552.199416:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401180 x1853163991638656/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318552 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318552.199437:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318552.199467:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318552.199472:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991638528:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318552.199479:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318552.199486:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318552.199489:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991638656:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767318552.199686:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318552.199693:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318552.199696:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318552.199711:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3726 00000100:00080000:3.0:1767318552.199738:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318552.199744:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318552.199775:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572aa00 x1853163991638784/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318552.199801:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318552.199811:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318552.199813:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318552.199816:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3726 00000100:00080000:3.0:1767318552.199819:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318552.199823:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318552.199832:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318552.199844:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729500 x1853163991638912/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318552.203066:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991638784:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318552.203233:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991638912:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767318552.391493:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318552.455224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318553.415867:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318553.479253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318554.439341:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318554.503219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318555.463671:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318555.530627:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318556.488807:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318556.551219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318557.321023:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318557.321042:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318557.321046:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318557.321049:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318557.516493:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318557.575371:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318558.535691:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318558.599316:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318559.564483:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318559.629011:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318560.583668:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318560.647249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318561.610260:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318561.671299:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318562.441351:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318562.441363:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318562.441365:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318562.441368:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318562.631653:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318562.695194:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318563.655900:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318563.719248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318564.686035:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318564.743604:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318565.706899:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318565.767173:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318566.728949:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318566.791218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318567.560037:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318567.560048:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318567.560125:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318567.560129:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318567.751473:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318567.815220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318568.775386:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318568.840612:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318569.799421:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318569.863828:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318570.823258:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318570.887226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318571.847600:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318571.911572:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318572.679232:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318572.679243:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318572.679246:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318572.679250:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318572.873015:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318572.935244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318573.895922:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318573.959220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318574.919524:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318574.983226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318575.945119:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318576.007216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318576.967473:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318577.031241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318577.799611:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318577.799624:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318577.799628:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318577.799631:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767318577.799727:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318552/real 1767318552] req@ffff88c5c572aa00 x1853163991638784/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318577 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318577.799763:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572aa00 x1853163991638784/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318577 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318577.799774:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318552/real 1767318552] req@ffff88c5c5729500 x1853163991638912/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318577 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318577.799786:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729500 x1853163991638912/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318577 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318577.799809:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318577.799830:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318577.799857:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991638784:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318577.799873:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318577.799878:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318577.799881:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991638912:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318577.799952:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318577.799956:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318577.799959:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318577.799965:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3751 00000100:00080000:2.0:1767318577.799976:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318577.799983:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318577.800016:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6300 x1853163991639040/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318577.800078:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991639040:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767318577.800263:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318577.800267:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318577.800269:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318577.800273:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3751 00000100:00080000:2.0:1767318577.800277:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318577.800281:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318577.800292:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767318577.800304:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7800 x1853163991639168/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318577.801439:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991639168:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318577.993060:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318578.055230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318579.015377:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318579.079228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318580.040402:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318580.103459:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318581.069574:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318581.127220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318582.087314:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318582.151210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318582.919718:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318582.919729:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318582.919732:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318582.919736:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318583.111299:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318583.176385:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318584.135846:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318584.199212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318585.161815:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318585.223835:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318586.185195:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318586.247242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318587.208172:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318587.272110:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318588.042690:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318588.042701:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318588.042704:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318588.042708:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318588.231603:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318588.295256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318589.255874:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318589.319339:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318590.279314:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318590.343240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318591.303462:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318591.367237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318592.330218:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318592.391218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318593.160612:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318593.160623:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318593.160626:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318593.160630:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318593.351960:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318593.415302:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318594.375685:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318594.439201:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318595.399725:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318595.463238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318596.423306:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318596.488186:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318597.447569:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318597.511244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318598.279797:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318598.279807:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318598.279809:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318598.279813:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318598.475036:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318598.540456:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318599.497765:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318599.559217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318600.520435:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318600.584019:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318601.543707:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318601.608223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318602.375277:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318577/real 1767318577] req@ffff88c5c57d7800 x1853163991639168/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318602 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318602.375311:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7800 x1853163991639168/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318602 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318602.375350:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318577/real 1767318577] req@ffff88c5c57d6300 x1853163991639040/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318602 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318602.375365:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6300 x1853163991639040/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318602 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318602.375384:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318602.375417:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318602.375422:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991639168:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767318602.375428:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318602.375434:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318602.375437:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991639040:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318602.375465:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318602.375470:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318602.375473:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318602.375478:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3776 00000100:00080000:3.0:1767318602.375486:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318602.375492:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318602.375523:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729500 x1853163991639296/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318602.375555:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318602.375559:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318602.375561:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318602.375565:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3776 00000100:00080000:3.0:1767318602.375569:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318602.375574:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318602.375583:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318602.375595:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572aa00 x1853163991639424/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318602.376507:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991639296:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318602.376603:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991639424:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318602.567506:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318602.631266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318603.591454:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318603.655315:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318604.616126:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318604.679324:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318605.640543:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318605.705039:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318606.664734:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318606.728119:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318607.495195:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318607.495207:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318607.495210:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318607.495213:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318607.687342:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318607.751886:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318608.711690:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318608.775231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318609.736844:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318609.799284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318610.761332:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318610.824017:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318611.783410:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318611.847339:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318612.615271:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318612.615284:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318612.615287:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318612.615291:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318612.807491:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318612.871488:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318613.835428:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318613.895206:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318614.857835:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318614.919311:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318615.882250:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318615.943364:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318616.905360:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318616.967280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318617.735435:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318617.735446:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318617.735449:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318617.735452:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318617.928177:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318617.991270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318618.951389:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318619.015223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318619.976404:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318620.039223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318621.002082:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318621.063189:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318622.026073:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318622.087324:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318622.855443:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318622.855453:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318622.855456:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318622.855459:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318623.050506:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318623.111265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318624.071757:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318624.135215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318625.096139:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318625.159239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318626.125215:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318626.183216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318627.143358:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318627.207239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318627.975539:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318602/real 1767318602] req@ffff88c5c5729500 x1853163991639296/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318627 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318627.975571:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729500 x1853163991639296/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318627 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318627.975581:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318602/real 1767318602] req@ffff88c5c572aa00 x1853163991639424/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318627 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318627.975593:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572aa00 x1853163991639424/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318627 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318627.975612:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318627.975632:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318627.975636:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991639296:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318627.975643:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318627.975647:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318627.975650:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991639424:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318627.975925:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318627.975930:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318627.975933:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318627.975938:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3801 00000100:00080000:2.0:1767318627.975947:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318627.975953:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318627.975983:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6a00 x1853163991639552/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767318627.976007:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318627.976010:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318627.976012:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318627.976016:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3801 00000100:00080000:2.0:1767318627.976020:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318627.976024:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318627.976033:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767318627.976040:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991639552:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767318627.976055:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4000 x1853163991639680/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318627.976150:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991639680:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767318628.167594:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318628.231293:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318629.191339:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318629.255277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318630.216408:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318630.279492:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318631.240668:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318631.305423:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318632.263720:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318632.327292:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318633.096942:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318633.096953:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318633.096956:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318633.096959:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318633.287420:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318633.351239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318634.311300:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318634.375227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318635.335339:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318635.399222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318636.359307:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318636.425197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318637.383426:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318637.448235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318638.215359:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318638.215369:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318638.215372:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318638.215376:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318638.407385:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318638.471200:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318639.431391:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318639.495452:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318640.456789:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318640.519223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318641.483918:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318641.543218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318642.503315:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318642.567489:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318643.335496:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318643.335506:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318643.335509:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318643.335512:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318643.536235:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318643.593222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318644.552671:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318644.615207:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318645.575307:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318645.639565:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318646.600064:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318646.663614:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318647.630890:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318647.687369:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318648.455258:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318648.455268:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318648.455271:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318648.455274:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318648.648479:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318648.711252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318649.671419:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318649.735215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318650.695374:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318650.760295:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318651.719540:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318651.783414:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767318652.551234:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318627/real 1767318627] req@ffff88c5c57d4000 x1853163991639680/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318652 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318652.551267:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4000 x1853163991639680/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318652 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318652.551278:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318627/real 1767318627] req@ffff88c5c57d6a00 x1853163991639552/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318652 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318652.551289:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6a00 x1853163991639552/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318652 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318652.551307:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318652.551353:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318652.551357:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991639680:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767318652.551363:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318652.551369:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318652.551372:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991639552:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318652.551607:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318652.551612:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318652.551616:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318652.551621:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3827 00000100:00080000:3.0:1767318652.551631:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318652.551637:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318652.551667:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729c00 x1853163991639808/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318652.551692:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318652.551695:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318652.551697:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318652.551701:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3827 00000100:00080000:3.0:1767318652.551704:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318652.551708:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318652.551718:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318652.551729:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729f80 x1853163991639936/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318652.552614:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991639808:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318652.552716:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991639936:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767318652.743713:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318652.810901:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318653.768150:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318653.831263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318654.792219:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318654.857250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318655.815343:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318655.879252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318656.840117:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318656.903374:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318657.672413:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318657.672429:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318657.672433:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318657.672437:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318657.863535:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318657.927678:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318658.887403:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318658.951271:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318659.913718:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318659.975265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318660.937139:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318660.999310:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318661.959659:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318662.025504:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318662.793958:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318662.793969:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318662.793972:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318662.793976:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318662.985691:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318663.047756:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318664.007685:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318664.073179:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318665.031711:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318665.096326:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318666.059918:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318666.119850:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318667.079471:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318667.144454:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318667.911735:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318667.911746:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318667.911765:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318667.911768:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318668.103332:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318668.167842:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318669.127278:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318669.191594:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318670.151466:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318670.215706:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318671.175291:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318671.241073:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318672.203858:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318672.263250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318673.033639:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318673.033652:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318673.033654:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318673.034228:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318673.224175:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318673.287205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318674.247338:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318674.311229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318675.279923:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318675.335409:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318676.295458:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318676.360157:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767318677.127453:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318652/real 1767318652] req@ffff88c5c5729c00 x1853163991639808/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318677 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318677.127487:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729c00 x1853163991639808/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318677 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318677.127499:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318652/real 1767318652] req@ffff88c5c5729f80 x1853163991639936/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318677 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318677.127510:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729f80 x1853163991639936/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318677 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318677.127529:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318677.127575:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318677.127580:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991639808:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318677.127586:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318677.127591:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318677.127594:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991639936:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318677.129047:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318677.129056:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318677.129059:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318677.129067:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3851 00000100:00080000:2.0:1767318677.129091:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318677.129129:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318677.129169:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5f80 x1853163991640064/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767318677.129239:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991640064:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767318677.129358:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318677.129362:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318677.129364:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318677.129369:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3851 00000100:00080000:2.0:1767318677.129373:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318677.129377:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318677.129386:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767318677.129399:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7480 x1853163991640192/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767318677.129585:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991640192:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767318677.319388:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318677.383343:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318678.348584:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318678.407199:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318679.367819:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318679.431348:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318680.391658:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318680.455299:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318681.415497:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318681.479258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318682.247413:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318682.247428:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318682.247431:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318682.247434:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318682.439609:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318682.505789:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318683.463295:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318683.527821:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318684.487427:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318684.551245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318685.511391:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318685.576068:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318686.542225:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318686.599300:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318687.367815:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318687.367825:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318687.367846:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318687.367850:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318687.559386:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318687.624051:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318688.583394:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318688.647215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318689.607623:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318689.671215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318690.631344:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318690.696675:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318691.656549:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318691.724256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318692.488606:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318692.488616:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318692.488619:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318692.488622:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318692.682299:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318692.743472:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318693.704924:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318693.767218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318694.727435:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318694.792902:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318695.751310:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318695.817469:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318696.777368:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318696.840775:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318697.607264:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318697.607277:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318697.607280:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318697.607283:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318697.808082:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318697.864428:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318698.824262:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318698.887679:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318699.847332:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318699.911291:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318700.871361:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318700.935692:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318701.895365:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318701.959311:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318702.727335:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318702.727346:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318702.727349:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318702.727352:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767318702.727434:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318677/real 1767318677] req@ffff88c5c57d7480 x1853163991640192/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318702 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318702.727935:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7480 x1853163991640192/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318702 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318702.727947:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318677/real 1767318677] req@ffff88c5c57d5f80 x1853163991640064/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318702 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318702.727962:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5f80 x1853163991640064/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318702 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318702.727981:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318702.728002:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318702.728006:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991640192:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767318702.728010:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318702.728015:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318702.728018:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991640064:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318702.728249:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318702.728254:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767318702.728257:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318702.728262:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3876 00000100:00080000:0.0:1767318702.728270:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318702.728276:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767318702.728305:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402680 x1853163991640320/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767318702.728328:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318702.728331:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767318702.728333:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767318702.728336:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3876 00000100:00080000:0.0:1767318702.728340:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767318702.728343:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767318702.728351:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767318702.728363:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400e00 x1853163991640448/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767318702.728949:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991640320:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767318702.729129:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991640448:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318702.919427:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318702.984252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318703.948049:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318704.007754:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318704.967452:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318705.031291:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318705.991617:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318706.055298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318707.015377:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318707.079278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318707.847754:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318707.847768:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318707.847772:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318707.847776:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318708.048681:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318708.103277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318709.063433:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318709.128040:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318710.087611:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318710.152149:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318711.111838:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318711.175294:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318712.135794:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318712.199644:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318712.978621:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318712.978635:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318712.978666:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318712.978670:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318713.159607:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318713.224661:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318714.185845:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318714.247611:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318715.210161:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318715.271218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318716.231463:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318716.297119:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318717.255449:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318717.319307:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318718.087319:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318718.087330:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318718.087333:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318718.087336:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318718.279515:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318718.343386:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318719.303432:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318719.367273:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318720.333639:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318720.391201:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318721.351739:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318721.415234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318722.375380:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318722.439723:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318723.207409:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318723.207419:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318723.207422:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318723.207425:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318723.399305:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318723.463244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318724.426966:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318724.488856:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318725.451511:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318725.511722:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318726.479256:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318726.536728:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767318727.303559:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318702/real 1767318702] req@ffff88c5c6402680 x1853163991640320/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318727 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318727.303596:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402680 x1853163991640320/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318727 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318727.303607:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318702/real 1767318702] req@ffff88c5c6400e00 x1853163991640448/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318727 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318727.303619:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400e00 x1853163991640448/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318727 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318727.303854:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318727.304039:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318727.304044:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991640320:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318727.304050:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318727.307290:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318727.307310:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767318727.307314:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318727.307398:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3901 00000100:00080000:1.0:1767318727.307409:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318727.307416:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767318727.307454:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf100 x1853163991640576/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767318727.307525:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318727.307546:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767318727.307548:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318727.307552:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3901 00000100:00080000:1.0:1767318727.307557:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318727.307561:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767318727.307570:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767318727.307583:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd880 x1853163991640704/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318727.308586:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318727.308608:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991640448:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767318727.308735:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991640576:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767318727.309572:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991640704:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318727.496695:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318727.559214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318728.523544:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318728.583196:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318729.543345:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318729.609426:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318730.568993:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318730.631524:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318731.591387:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318731.655243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318732.423218:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318732.423237:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318732.423240:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318732.423243:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318732.615554:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318732.679243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318733.639413:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318733.703277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318734.663823:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318734.727321:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318735.689114:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318735.751269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318736.711698:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318736.777692:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318737.543368:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318737.543378:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318737.543382:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318737.543385:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318737.736690:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318737.799226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318738.759546:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318738.828134:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318739.783545:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318739.847427:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318740.810781:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318740.871223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318741.832151:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318741.895232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318742.663763:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318742.663775:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318742.663778:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318742.663781:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318742.855854:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318742.919331:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318743.880568:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318743.943338:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318744.906742:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318744.968162:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318745.929009:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318745.992626:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318746.951390:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318747.015351:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318747.783570:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318747.783580:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318747.783583:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318747.783586:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318747.975386:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318748.040042:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318749.001942:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318749.063444:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318750.024262:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318750.087531:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318751.050723:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318751.112689:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318752.071667:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318752.135275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318752.903335:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318752.903343:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318752.903346:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1767318752.903347:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318727/real 1767318727] req@ffff88c5c65cf100 x1853163991640576/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318752 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318752.903350:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767318752.903382:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf100 x1853163991640576/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318752 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318752.903392:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318727/real 1767318727] req@ffff88c5c65cd880 x1853163991640704/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318752 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767318752.903429:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd880 x1853163991640704/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318752 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767318752.903447:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318752.903467:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318752.903471:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991640576:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767318752.903477:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767318752.903481:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767318752.903485:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991640704:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767318752.903517:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318752.903536:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318752.903540:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318752.903545:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3926 00000100:00080000:3.0:1767318752.903554:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318752.903560:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318752.903589:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729f80 x1853163991640832/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318752.903613:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318752.903616:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318752.903618:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318752.903621:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3926 00000100:00080000:3.0:1767318752.903625:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318752.903629:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318752.903637:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318752.903649:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729c00 x1853163991640960/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767318752.905599:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991640832:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767318752.905701:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991640960:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767318753.095362:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318753.159822:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318754.119650:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318754.184770:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318755.144380:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318755.207219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318756.170033:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318756.232864:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318757.192901:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318757.255404:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318758.023258:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318758.023271:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318758.023298:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318758.023302:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318758.215372:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318758.279258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318759.242568:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318759.303412:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318760.263329:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318760.327285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318761.287435:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318761.352774:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318762.311880:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318762.375880:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318763.145564:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318763.145575:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318763.145578:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318763.145581:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318763.338693:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318763.399285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318764.359655:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318764.423220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318765.383536:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318765.447403:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318766.407654:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318766.471327:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318767.432402:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318767.495200:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318768.263407:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318768.263418:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318768.263421:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318768.263424:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318768.455391:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318768.521410:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767318769.479540:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318769.543440:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318770.503550:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318770.567260:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318771.527619:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318771.591220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318772.551344:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318772.615311:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318773.383701:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318773.383714:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318773.383717:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318773.383720:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767318773.575563:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318773.639301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318774.599612:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318774.663344:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318775.624148:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318775.688530:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318776.649218:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318776.711239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767318777.479497:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318752/real 1767318752] req@ffff88c5c5729f80 x1853163991640832/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318777 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318777.479535:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729f80 x1853163991640832/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318777 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318777.479548:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318752/real 1767318752] req@ffff88c5c5729c00 x1853163991640960/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318777 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318777.479561:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729c00 x1853163991640960/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318777 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318777.479584:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318777.479624:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318777.479629:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991640832:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318777.479637:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318777.479644:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318777.479647:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991640960:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767318777.479954:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318777.479960:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767318777.479964:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318777.479970:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3952 00000100:00080000:1.0:1767318777.479981:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318777.480008:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767318777.480046:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd880 x1853163991641088/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767318777.480160:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318777.480164:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767318777.480167:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767318777.480171:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3952 00000100:00080000:1.0:1767318777.480176:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767318777.480181:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767318777.480191:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767318777.480204:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf100 x1853163991641216/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767318777.480423:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991641088:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767318777.480552:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991641216:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318777.671574:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318777.735231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318778.695265:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318778.759257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318779.719360:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318779.784838:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318780.743343:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318780.807255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318781.767386:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318781.831218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318782.599402:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318782.599414:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318782.599417:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318782.599420:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:0.0:1767318782.791437:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318782.855240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318783.815411:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318783.879710:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318784.839420:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318784.903248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318785.863268:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318785.927306:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318786.888179:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318786.951713:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318787.724211:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318787.724224:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318787.724227:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318787.724231:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318787.911656:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318787.976464:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318788.946595:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318788.999242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318789.961859:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318790.023273:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318790.983373:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318791.047776:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318792.007364:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318792.071261:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318792.841046:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318792.841057:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318792.841061:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318792.841064:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318793.031316:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318793.095202:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318794.058597:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318794.119360:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318795.079265:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318795.143381:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318796.103354:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318796.170079:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318797.127486:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318797.192688:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318797.959354:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318797.959366:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318797.959369:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318797.959373:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318798.152462:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318798.215412:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318799.175835:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318799.239243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318800.200463:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318800.266727:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318801.223358:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318801.292747:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767318802.055896:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318777/real 1767318777] req@ffff88c5c65cd880 x1853163991641088/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318802 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318802.055937:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd880 x1853163991641088/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318802 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318802.055948:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318777/real 1767318777] req@ffff88c5c65cf100 x1853163991641216/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318802 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767318802.055960:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf100 x1853163991641216/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318802 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767318802.055980:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318802.056009:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318802.056014:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991641088:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318802.056020:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318802.056025:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767318802.056028:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991641216:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318802.056224:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318802.056257:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318802.056260:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318802.056267:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 3976 00000100:00080000:2.0:1767318802.056276:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318802.056282:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318802.056314:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4a80 x1853163991641344/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767318802.056340:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318802.056343:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318802.056345:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318802.056348:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 3976 00000100:00080000:2.0:1767318802.056351:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318802.056354:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318802.056363:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767318802.056374:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5c00 x1853163991641472/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767318802.056755:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991641344:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767318802.056888:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991641472:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318802.247327:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318802.311296:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318803.271416:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318803.335798:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318804.296784:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318804.359238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318805.319278:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318805.383190:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318806.344197:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318806.407225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318807.175850:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318807.175862:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318807.175865:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318807.175868:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:0.0:1767318807.367272:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318807.432268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318808.395685:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318808.455382:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318809.415428:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318809.480854:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318810.439336:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318810.504584:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318811.463261:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318811.527335:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318812.295364:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318812.295381:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318812.295384:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318812.295388:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318812.487356:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318812.551183:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318813.511650:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318813.576756:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318814.535332:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318814.599230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318815.559886:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318815.624048:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318816.583850:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318816.647226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318817.416126:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318817.416137:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318817.416141:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318817.416144:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318817.607572:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318817.673778:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318818.631437:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318818.695400:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318819.655346:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318819.719253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318820.679626:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318820.745783:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318821.703686:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318821.767809:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318822.535859:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318822.535874:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318822.535877:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318822.535881:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318822.727474:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318822.791287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318823.751347:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318823.815325:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318824.775801:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318824.841527:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318825.800117:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318825.863229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318826.823376:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318826.887259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318827.655276:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318802/real 1767318802] req@ffff88c5c57d4a80 x1853163991641344/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318827 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318827.655312:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4a80 x1853163991641344/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318827 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318827.655322:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318802/real 1767318802] req@ffff88c5c57d5c00 x1853163991641472/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318827 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318827.655334:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5c00 x1853163991641472/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318827 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318827.655353:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318827.655387:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318827.655391:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991641344:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318827.655424:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318827.655432:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318827.655435:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991641472:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318827.655532:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318827.655538:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318827.655541:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318827.655548:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4001 00000100:00080000:2.0:1767318827.655559:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318827.655567:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318827.655600:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4380 x1853163991641600/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767318827.655635:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318827.655639:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318827.655641:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318827.655645:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4001 00000100:00080000:2.0:1767318827.655649:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318827.655654:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318827.655662:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767318827.655675:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5880 x1853163991641728/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318827.655752:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991641600:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318827.655848:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991641728:192.168.204.145@tcp:38:kworker.0 00010000:00010000:0.0:1767318827.847270:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318827.911556:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318828.873451:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318828.936054:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318829.895539:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318829.959230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318830.919505:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318830.984372:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318831.943333:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318832.007320:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318832.780503:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318832.780517:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318832.780528:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318832.780532:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318832.967502:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318833.031807:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318833.994990:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318834.055699:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318835.015789:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318835.079925:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767318836.040481:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318836.103216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318837.063349:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318837.127222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318837.896191:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318837.896201:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318837.896205:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318837.896226:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318838.087765:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318838.151469:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318839.115425:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318839.175215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318840.140046:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318840.199544:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318841.161318:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318841.223325:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318842.183841:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318842.247660:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318843.015338:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318843.015350:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318843.015353:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318843.015356:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318843.207575:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318843.272970:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318844.231391:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318844.295228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318845.258185:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318845.333567:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318846.280346:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318846.343549:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318847.303791:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318847.367298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318848.135359:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318848.135370:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318848.135373:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318848.135376:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767318848.327556:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318848.391254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318849.352175:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318849.415266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318850.376075:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318850.442471:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318851.401208:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318851.465728:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318852.231516:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318827/real 1767318827] req@ffff88c5c57d4380 x1853163991641600/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318852 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318852.231555:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4380 x1853163991641600/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318852 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318852.231565:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318827/real 1767318827] req@ffff88c5c57d5880 x1853163991641728/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318852 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318852.231598:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5880 x1853163991641728/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318852 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318852.231616:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318852.231646:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318852.231650:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991641600:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318852.231657:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318852.231672:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318852.231676:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991641728:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767318852.232768:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318852.232784:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318852.232788:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318852.232795:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4026 00000100:00080000:3.0:1767318852.232805:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318852.232821:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318852.232852:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729c00 x1853163991641856/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318852.232879:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318852.232882:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318852.232884:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318852.232888:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4026 00000100:00080000:3.0:1767318852.232891:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318852.232894:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318852.232903:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318852.232913:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729f80 x1853163991641984/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318852.233018:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991641856:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318852.233143:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991641984:192.168.204.145@tcp:38:kworker.0 00010000:00010000:2.0:1767318852.423296:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318852.488859:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318853.447332:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318853.511336:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318854.471736:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318854.535756:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318855.495859:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318855.559227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318856.521140:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318856.584932:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318857.351571:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318857.351582:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318857.351585:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318857.351588:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318857.544080:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318857.607299:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318858.567311:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318858.631234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318859.591328:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318859.655548:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318860.617931:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318860.680482:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318861.642083:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318861.703318:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318862.473588:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318862.473599:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318862.473622:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318862.473625:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318862.663375:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318862.728015:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318863.688562:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318863.751396:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318864.711685:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318864.775385:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318865.738337:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318865.799568:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318866.759521:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318866.823215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318867.591256:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318867.591267:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318867.591270:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318867.591273:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318867.783589:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318867.847228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318868.807696:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318868.871243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318869.833577:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318869.895703:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318870.855779:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318870.919334:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318871.880589:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318871.943555:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318872.712493:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318872.712504:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318872.712507:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318872.712510:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767318872.906337:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318872.967186:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318873.928000:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318873.992416:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318874.951382:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318875.015244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318875.975297:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318876.039249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318876.999411:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318877.063497:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318877.831275:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318877.831285:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318877.831288:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318877.831291:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767318877.832603:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318852/real 1767318852] req@ffff88c5c5729c00 x1853163991641856/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318877 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318877.832636:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729c00 x1853163991641856/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318877 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318877.832647:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318852/real 1767318852] req@ffff88c5c5729f80 x1853163991641984/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318877 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318877.832659:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729f80 x1853163991641984/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318877 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318877.832677:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318877.832754:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318877.832759:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991641856:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318877.832766:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767318877.833741:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318877.833749:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318877.833753:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318877.833759:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4051 00000100:00080000:3.0:1767318877.833786:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318877.833793:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318877.833826:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572ad80 x1853163991642112/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318877.833841:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318877.833844:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318877.833846:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318877.833850:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4051 00000100:00080000:3.0:1767318877.833853:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318877.833856:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318877.834071:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318877.834085:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728000 x1853163991642240/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767318877.834280:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318877.834286:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991641984:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767318877.834317:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991642112:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318877.834781:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991642240:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318878.023457:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318878.087238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318879.047441:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318879.111214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318880.071778:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318880.135243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318881.095372:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318881.159256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318882.119992:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318882.185856:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318882.951520:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318882.951532:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318882.951535:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318882.951539:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318883.143710:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318883.207227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318884.167680:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318884.231956:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318885.194735:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318885.255289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318886.215475:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318886.279234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318887.242664:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318887.306306:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318888.071423:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318888.071433:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318888.071437:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318888.071459:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318888.263698:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318888.328065:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318889.289234:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318889.351227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318890.311916:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318890.375229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318891.337550:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318891.399223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318892.359367:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318892.423229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318893.193837:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318893.193849:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318893.193852:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318893.193856:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767318893.383457:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318893.447222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318894.407322:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318894.471164:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318895.431903:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318895.495217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318896.456248:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318896.519240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318897.479376:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318897.543651:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318898.311248:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318898.311268:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318898.311271:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318898.311274:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318898.504127:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318898.567297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318899.527303:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318899.591268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318900.551436:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318900.615367:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318901.578072:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318901.639530:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318902.407283:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318877/real 1767318877] req@ffff88c5c572ad80 x1853163991642112/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318902 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318902.407356:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572ad80 x1853163991642112/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318902 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318902.407369:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318877/real 1767318877] req@ffff88c5c5728000 x1853163991642240/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318902 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318902.407381:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728000 x1853163991642240/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318902 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318902.407403:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318902.407436:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318902.407441:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991642112:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318902.407448:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318902.407455:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318902.407458:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991642240:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767318902.408709:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318902.408730:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318902.408735:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318902.408745:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4077 00000100:00080000:3.0:1767318902.408756:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318902.408763:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318902.408800:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729180 x1853163991642368/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318902.408904:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991642368:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318902.410511:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318902.410520:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318902.410523:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318902.410530:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4077 00000100:00080000:3.0:1767318902.410781:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318902.410788:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318902.410799:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318902.410822:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b100 x1853163991642496/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318902.414889:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991642496:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318902.602474:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318902.663250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318903.624419:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318903.687408:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318904.648143:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318904.711239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318905.673367:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318905.735228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318906.695483:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318906.759404:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318907.527294:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318907.527306:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318907.527308:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318907.527311:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318907.719413:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318907.784256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318908.744723:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318908.807419:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318909.767736:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318909.831331:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318910.793549:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318910.855481:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318911.815674:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318911.879308:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318912.647194:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318912.647206:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318912.647228:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318912.647232:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318912.841278:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318912.904763:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318913.863435:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318913.927433:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318914.887783:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318914.953160:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318915.913553:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318915.975751:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318916.935348:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318916.999277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318917.767463:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318917.767474:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318917.767477:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318917.767480:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767318917.962520:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318918.023230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318918.986466:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318919.047238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318920.008029:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318920.075054:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318921.034425:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318921.095221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318922.055525:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318922.120239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318922.890423:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318922.890433:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318922.890453:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318922.890457:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318923.079989:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318923.143645:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318924.110390:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318924.167878:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318925.130256:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318925.191214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318926.162319:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318926.216080:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318927.175313:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318927.241420:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318928.007210:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318902/real 1767318902] req@ffff88c5c572b100 x1853163991642496/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318927 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318928.007257:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b100 x1853163991642496/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318927 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318928.007269:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318902/real 1767318902] req@ffff88c5c5729180 x1853163991642368/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318927 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318928.007306:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729180 x1853163991642368/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318927 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318928.007325:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318928.007351:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318928.007356:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991642496:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767318928.007362:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318928.007368:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318928.007371:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991642368:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318928.008309:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318928.008317:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318928.008320:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318928.008328:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4101 00000100:00080000:3.0:1767318928.008338:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318928.008345:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318928.008378:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728700 x1853163991642624/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318928.008408:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318928.008411:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318928.008413:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318928.008417:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4101 00000100:00080000:3.0:1767318928.008421:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318928.008433:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318928.008443:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318928.008455:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728380 x1853163991642752/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318928.008642:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991642624:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318928.008756:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991642752:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318928.202067:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318928.263315:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318929.225294:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318929.287214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318930.247612:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318930.311241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318931.271483:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318931.335173:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318932.295402:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318932.362296:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318933.127218:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318933.127229:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318933.127232:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318933.127235:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318933.319931:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318933.383244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318934.343338:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318934.407232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318935.373244:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318935.431494:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318936.392125:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318936.455523:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318937.415354:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318937.479310:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318938.247261:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318938.247271:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318938.247274:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318938.247278:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767318938.440443:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318938.503326:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318939.466068:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318939.528263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318940.487314:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318940.552546:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318941.511309:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318941.575219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318942.535538:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318942.601815:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318943.367310:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318943.367321:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318943.367324:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318943.367327:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318943.559348:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318943.626174:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318944.583474:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318944.647251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318945.607405:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318945.671194:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318946.633754:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318946.695250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318947.655307:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318947.719226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318948.487630:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318948.487640:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318948.487643:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318948.487647:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318948.683764:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318948.743264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318949.704136:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318949.767275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318950.727531:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318950.791211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318951.751368:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318951.815224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318952.775362:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318952.839340:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318953.607327:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318928/real 1767318928] req@ffff88c5c5728700 x1853163991642624/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318953 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318953.607372:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728700 x1853163991642624/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318953 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318953.607384:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318928/real 1767318928] req@ffff88c5c5728380 x1853163991642752/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318953 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318953.607396:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728380 x1853163991642752/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318953 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318953.607419:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318953.607445:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318953.607481:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991642624:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767318953.607488:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318953.607494:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318953.607498:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991642752:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767318953.608564:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318953.608576:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767318953.608594:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318953.608602:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4127 00000100:00080000:2.0:1767318953.608616:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318953.608623:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767318953.608661:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7b80 x1853163991642880/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318953.608734:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991642880:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767318953.609085:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318953.609090:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767318953.609092:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767318953.609117:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4127 00000100:00080000:2.0:1767318953.609121:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767318953.609125:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767318953.609135:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767318953.609150:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5180 x1853163991643008/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318953.613601:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991643008:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318953.799776:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318953.863264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318954.823367:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318954.887271:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318955.847269:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318955.913027:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318956.871486:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318956.935590:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318957.897194:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318957.959224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318958.727486:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318958.727497:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318958.727500:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318958.727513:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767318958.920400:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318958.984479:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318959.943948:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318960.007293:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318960.967454:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318961.031486:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318961.991442:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318962.055285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318963.016873:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318963.079689:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318963.847394:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318963.847408:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318963.847412:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318963.847416:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318964.042200:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318964.104819:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318965.063626:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318965.127319:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318966.087480:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318966.152197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318967.111460:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318967.175262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318968.135782:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318968.199366:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318968.967454:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318968.967468:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318968.967481:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318968.967485:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318969.159380:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318969.223276:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318970.183427:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318970.249265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318971.208160:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318971.271974:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318972.231607:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318972.295559:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318973.255594:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318973.321133:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767318974.091472:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318974.091488:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767318974.091491:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318974.091495:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318974.279771:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318974.343263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318975.303348:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318975.367562:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318976.327355:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318976.392172:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318977.352843:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318977.416182:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767318978.183798:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318953/real 1767318953] req@ffff88c5c57d5180 x1853163991643008/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318978 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318978.183831:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5180 x1853163991643008/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767318978 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318978.183843:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318953/real 1767318953] req@ffff88c5c57d7b80 x1853163991642880/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318978 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767318978.183880:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7b80 x1853163991642880/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767318978 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767318978.183899:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318978.183928:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318978.183933:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991643008:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767318978.183939:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767318978.183945:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767318978.183948:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991642880:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767318978.184037:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318978.184044:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767318978.184047:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318978.184052:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4152 00000100:00080000:3.0:1767318978.184062:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318978.184068:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767318978.184134:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729880 x1853163991643136/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767318978.184161:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767318978.184164:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767318978.184173:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767318978.184177:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4152 00000100:00080000:3.0:1767318978.184181:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767318978.184185:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767318978.184195:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767318978.184208:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728e00 x1853163991643264/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767318978.184762:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991643136:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767318978.184854:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991643264:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767318978.375378:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318978.443381:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318979.399334:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318979.463223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318980.423443:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318980.487240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318981.447755:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318981.511997:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318982.471271:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318982.535675:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767318983.306166:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318983.306177:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767318983.306180:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767318983.306183:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767318983.495307:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318983.559242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318984.520008:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318984.584708:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318985.543540:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318985.607282:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318986.569721:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318986.631285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318987.594178:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318987.655245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318988.423251:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318988.423261:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318988.423264:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318988.423268:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318988.615329:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318988.679193:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318989.639308:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318989.703765:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318990.663533:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318990.727280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318991.687359:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318991.751220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318992.711347:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318992.776926:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767318993.543942:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318993.543952:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767318993.543955:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767318993.543958:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318993.736350:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318993.799275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318994.759540:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767318994.825828:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318995.783373:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318995.850348:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318996.807354:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318996.871258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767318997.831901:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318997.895226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767318998.663267:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318998.663275:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767318998.663277:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767318998.663280:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767318998.855294:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318998.920687:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767318999.879879:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767318999.943265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319000.903341:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319000.967199:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319001.927479:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319001.991246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319002.951350:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319003.015946:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319003.783237:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1767319003.783239:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318978/real 1767318978] req@ffff88c5c5729880 x1853163991643136/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319003 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319003.783245:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319003.783248:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319003.783261:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319003.783269:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729880 x1853163991643136/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319003 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319003.783278:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767318978/real 1767318978] req@ffff88c5c5728e00 x1853163991643264/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319003 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319003.783289:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728e00 x1853163991643264/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319003 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319003.783306:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319003.783327:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319003.783332:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991643136:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319003.783338:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319003.783361:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319003.783365:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991643264:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767319003.783440:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319003.783517:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319003.783529:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319003.783535:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4177 00000100:00080000:2.0:1767319003.783545:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319003.783551:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319003.783582:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4700 x1853163991643392/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319003.783666:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991643392:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319003.783773:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319003.783784:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319003.783786:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319003.783790:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4177 00000100:00080000:2.0:1767319003.783795:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319003.783799:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319003.783808:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319003.783850:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6d80 x1853163991643520/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319003.784191:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991643520:192.168.204.145@tcp:38:kworker.0 00010000:00010000:2.0:1767319003.975352:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319004.041375:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319004.999306:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319005.063301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319006.023281:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319006.087484:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319007.047423:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319007.111571:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319008.071428:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319008.135367:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319008.903294:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319008.903305:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319008.903308:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319008.903312:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319009.095442:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319009.159218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319010.119330:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319010.183233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319011.143381:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319011.207211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319012.167347:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319012.231258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319013.191301:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319013.255852:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319014.023562:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319014.023573:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319014.023576:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319014.023579:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319014.215966:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319014.279228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319015.240183:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319015.304073:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319016.263302:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319016.327862:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319017.287337:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319017.351964:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319018.315136:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319018.375202:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319019.146702:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319019.146735:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319019.146738:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319019.146741:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319019.337708:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319019.399699:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319020.361575:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319020.424430:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319021.384240:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319021.447343:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319022.407811:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319022.471723:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319023.431409:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319023.495251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319024.263236:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319024.263246:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319024.263249:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319024.263252:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319024.455368:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319024.519218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319025.479332:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319025.543814:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319026.503713:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319026.567184:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319027.531572:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319027.592066:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319028.359207:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319003/real 1767319003] req@ffff88c5c57d6d80 x1853163991643520/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319028 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319028.359239:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6d80 x1853163991643520/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319028 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319028.359250:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319003/real 1767319003] req@ffff88c5c57d4700 x1853163991643392/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319028 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319028.359260:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4700 x1853163991643392/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319028 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319028.359278:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319028.359299:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319028.359303:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991643520:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319028.359334:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319028.359340:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319028.359344:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991643392:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319028.361191:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319028.361213:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319028.361216:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319028.361224:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4202 00000100:00080000:2.0:1767319028.361234:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319028.361242:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319028.361274:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4700 x1853163991643648/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767319028.361301:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319028.361305:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319028.361307:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319028.361310:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4202 00000100:00080000:2.0:1767319028.361314:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319028.361318:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319028.361327:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319028.361337:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6d80 x1853163991643776/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319028.361663:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991643648:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767319028.361772:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991643776:192.168.204.145@tcp:38:kworker.0 00010000:00010000:2.0:1767319028.551375:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319028.615240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319029.575432:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319029.639242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319030.600865:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319030.663819:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319031.625986:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319031.687234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319032.647678:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319032.711292:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319033.479247:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319033.479258:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319033.479261:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319033.479275:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319033.671315:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319033.735288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319034.695564:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319034.759957:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319035.719405:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319035.783184:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319036.745084:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319036.807236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319037.769719:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319037.831231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319038.599340:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319038.599351:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319038.599354:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319038.599357:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319038.791579:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319038.855222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319039.815507:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319039.879238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319040.840988:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319040.904435:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319041.864430:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319041.927305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319042.887385:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319042.951239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319043.719345:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319043.719356:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319043.719359:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319043.719363:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319043.911405:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319043.975248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319044.936189:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319044.999179:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319045.960055:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319046.024084:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319046.983788:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319047.047230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319048.007831:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319048.071218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319048.839251:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319048.839262:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319048.839265:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319048.839269:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319049.032027:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319049.095234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319050.055534:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319050.119670:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319051.079394:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319051.143302:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319052.103440:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319052.170129:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319053.127315:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319053.191529:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319053.959212:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319028/real 1767319028] req@ffff88c5c57d4700 x1853163991643648/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319053 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319053.959246:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4700 x1853163991643648/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319053 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319053.959279:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319028/real 1767319028] req@ffff88c5c57d6d80 x1853163991643776/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319053 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319053.959293:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6d80 x1853163991643776/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319053 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319053.959312:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319053.959344:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319053.959349:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991643648:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319053.959355:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319053.959361:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319053.959363:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991643776:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319053.959458:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319053.959464:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767319053.959467:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319053.959473:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4227 00000100:00080000:0.0:1767319053.959481:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319053.959488:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767319053.959519:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403100 x1853163991643904/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319053.959545:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319053.959548:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767319053.959549:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319053.959553:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4227 00000100:00080000:0.0:1767319053.959557:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319053.959560:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767319053.959569:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767319053.959581:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403b80 x1853163991644032/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319053.960078:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991643904:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767319053.960224:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991644032:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319054.152799:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319054.215889:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319055.175337:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319055.244931:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319056.204513:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319056.263236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319057.224888:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319057.287316:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319058.247621:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319058.311237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319059.079358:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319059.079368:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319059.079371:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319059.079376:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319059.271320:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319059.335283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319060.295313:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319060.360326:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319061.320048:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319061.383341:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319062.343772:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319062.407852:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319063.367304:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319063.431486:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319064.199296:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319064.199310:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319064.199314:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319064.199339:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319064.391507:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319064.455234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319065.415850:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319065.479238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319066.439321:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319066.503215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319067.465902:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319067.527218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319068.488118:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319068.551233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319069.319511:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319069.319532:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319069.319535:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319069.319538:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319069.512271:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319069.575833:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319070.537669:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319070.599227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319071.559390:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319071.623779:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319072.583301:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319072.647254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319073.607416:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319073.671272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319074.439390:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319074.439402:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319074.439422:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319074.439425:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319074.632857:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319074.697773:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319075.655448:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319075.721710:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319076.683362:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319076.744501:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319077.704174:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319077.768346:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319078.535386:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319053/real 1767319053] req@ffff88c5c6403100 x1853163991643904/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319078 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319078.535452:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403100 x1853163991643904/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319078 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319078.535463:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319053/real 1767319053] req@ffff88c5c6403b80 x1853163991644032/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319078 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319078.535476:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403b80 x1853163991644032/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319078 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319078.535493:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319078.535579:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319078.535584:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991643904:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319078.535591:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319078.536178:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319078.536188:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319078.536191:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319078.536197:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4253 00000100:00080000:2.0:1767319078.536205:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319078.536211:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319078.536246:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7b80 x1853163991644160/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767319078.536261:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319078.536264:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319078.536265:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319078.536269:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4253 00000100:00080000:2.0:1767319078.536272:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319078.536276:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319078.536284:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319078.536296:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5180 x1853163991644288/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319078.536680:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319078.536685:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991644032:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767319078.536716:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991644160:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319078.536817:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991644288:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319078.733326:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319078.791286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319079.751329:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319079.815237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319080.775363:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319080.839216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319081.800732:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319081.864187:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319082.823456:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319082.888854:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319083.655212:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319083.655225:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319083.655228:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319083.655232:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319083.847429:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319083.911280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319084.873119:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319084.935280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319085.899887:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319085.960521:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319086.921444:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319086.983703:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319087.943962:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319088.007262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319088.776394:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319088.776408:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319088.776411:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319088.776415:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319088.968481:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319089.031281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319089.993061:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319090.055299:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319091.015523:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319091.079640:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319092.042553:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319092.104476:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319093.065076:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319093.127507:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319093.895390:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319093.895404:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319093.895409:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319093.895412:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319094.087657:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319094.152139:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319095.112903:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319095.175270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319096.135993:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319096.200027:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319097.160135:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319097.223830:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319098.184140:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319098.247246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319099.016838:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319099.016849:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319099.016852:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319099.016855:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319099.207471:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319099.271265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319100.231585:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319100.295439:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319101.255481:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319101.319463:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319102.283134:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319102.346208:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319103.111953:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319078/real 1767319078] req@ffff88c5c57d7b80 x1853163991644160/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319103 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319103.112010:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7b80 x1853163991644160/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319103 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319103.112023:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319078/real 1767319078] req@ffff88c5c57d5180 x1853163991644288/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319103 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319103.112036:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5180 x1853163991644288/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319103 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767319103.112084:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319103.112128:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319103.112135:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991644160:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767319103.112145:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319103.112157:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319103.112168:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319103.112173:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991644288:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319103.112245:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767319103.112249:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319103.115248:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4277 00000100:00080000:0.0:1767319103.115278:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319103.115285:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767319103.115318:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403b80 x1853163991644416/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319103.115353:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319103.115357:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767319103.115360:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319103.115364:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4277 00000100:00080000:0.0:1767319103.115367:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319103.115370:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767319103.115385:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767319103.115396:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403100 x1853163991644544/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767319103.115675:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991644416:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767319103.116005:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991644544:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319103.305012:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319103.367219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319104.327328:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319104.391228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319105.351485:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319105.415297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319106.375626:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319106.439187:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319107.399379:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319107.463219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319108.232230:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319108.232242:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319108.232245:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319108.232248:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319108.425139:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319108.490369:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319109.447275:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319109.511183:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319110.471672:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319110.535832:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319111.495342:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319111.559256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319112.520009:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319112.583698:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319113.351869:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319113.351877:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319113.351879:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319113.351881:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319113.543358:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319113.607288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319114.568363:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319114.632471:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319115.591327:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319115.655227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319116.615739:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319116.679771:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319117.639542:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319117.703238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319118.471350:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319118.471360:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319118.471363:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319118.471367:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319118.663811:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319118.727231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319119.687596:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319119.753448:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319120.711328:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319120.777355:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319121.735660:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319121.800229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319122.759350:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319122.823251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319123.592306:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319123.592316:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319123.592319:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319123.592322:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319123.783878:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319123.847327:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319124.812423:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319124.872982:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319125.831347:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319125.895238:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319126.858066:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319126.920324:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319127.879532:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319127.943326:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319128.711386:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319128.711397:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319128.711399:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319128.711402:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767319128.712902:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319103/real 1767319103] req@ffff88c5c6403b80 x1853163991644416/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319128 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319128.712937:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403b80 x1853163991644416/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319128 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319128.712947:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319103/real 1767319103] req@ffff88c5c6403100 x1853163991644544/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319128 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319128.712959:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403100 x1853163991644544/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319128 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767319128.713004:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319128.713033:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319128.713037:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991644416:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767319128.713044:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319128.713050:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319128.713053:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991644544:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319128.713148:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319128.713153:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319128.713156:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319128.713181:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4302 00000100:00080000:3.0:1767319128.713192:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319128.713199:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319128.713229:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728e00 x1853163991644672/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319128.713260:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319128.713262:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319128.713264:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319128.713268:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4302 00000100:00080000:3.0:1767319128.713271:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319128.713275:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319128.713283:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319128.713294:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729880 x1853163991644800/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767319128.713752:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991644672:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767319128.713856:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991644800:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319128.905609:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319128.967239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319129.927655:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319129.991385:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319130.951407:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319131.015221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319131.976514:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319132.039377:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319132.999440:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319133.063225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319133.831225:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319133.831238:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319133.831242:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319133.831245:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319134.023307:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319134.087231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319135.047236:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319135.111241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319136.071508:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319136.135262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319137.095532:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319137.159230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319138.121197:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319138.183268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319138.951321:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319138.951329:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319138.951331:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319138.951334:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319139.143632:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319139.207241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319140.168499:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319140.231626:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319141.193605:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319141.255254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319142.215492:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319142.279275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319143.240824:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319143.303401:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319144.072742:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319144.072753:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319144.072755:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319144.072759:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319144.263342:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319144.327334:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319145.287377:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319145.351904:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319146.311593:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319146.375276:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319147.338320:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319147.399676:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319148.361393:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319148.423235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319149.192167:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319149.192179:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319149.192181:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319149.192184:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319149.384009:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319149.447234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319150.407630:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319150.471270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319151.431239:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319151.495336:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319152.455337:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319152.519234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319153.287840:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319128/real 1767319128] req@ffff88c5c5728e00 x1853163991644672/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319153 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319153.287876:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728e00 x1853163991644672/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319153 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319153.287915:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319128/real 1767319128] req@ffff88c5c5729880 x1853163991644800/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319153 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319153.287928:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729880 x1853163991644800/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319153 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767319153.287947:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319153.287976:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319153.287981:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991644672:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767319153.287987:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319153.287993:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319153.287996:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991644800:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319153.288117:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319153.288122:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767319153.288125:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319153.288131:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4327 00000100:00080000:0.0:1767319153.288141:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319153.288147:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767319153.288180:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401c00 x1853163991644928/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319153.288206:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319153.288209:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767319153.288217:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319153.288222:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4327 00000100:00080000:0.0:1767319153.288227:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319153.288231:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767319153.288240:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767319153.288253:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402d80 x1853163991645056/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767319153.288506:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991644928:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767319153.288866:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991645056:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319153.482785:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319153.543206:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319154.503872:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319154.567527:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319155.527474:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319155.591218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319156.551596:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319156.615431:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319157.575427:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319157.639551:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319158.407273:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319158.407285:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319158.407288:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319158.407292:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319158.599411:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319158.663171:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319159.623447:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319159.689428:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319160.647457:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319160.711268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319161.671438:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319161.735496:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319162.696473:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319162.763316:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319163.527326:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319163.527339:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319163.527342:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319163.527346:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319163.719460:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319163.783230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319164.744204:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319164.807286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319165.767375:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319165.831268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319166.791505:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319166.855563:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319167.816130:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319167.879807:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319168.647265:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319168.647276:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319168.647279:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319168.647282:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319168.839515:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319168.903220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319169.863537:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319169.927399:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319170.887277:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319170.951222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319171.912005:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319171.975356:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319172.935342:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319172.999528:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319173.767220:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319173.767230:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319173.767233:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319173.767236:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319173.962520:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319174.024065:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319174.983927:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319175.047233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319176.007411:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319176.071365:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319177.031317:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319177.095203:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319178.056227:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319178.119249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319178.887672:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319153/real 1767319153] req@ffff88c5c6401c00 x1853163991644928/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319178 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319178.887707:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401c00 x1853163991644928/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319178 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319178.887740:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319153/real 1767319153] req@ffff88c5c6402d80 x1853163991645056/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319178 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319178.887751:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402d80 x1853163991645056/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319178 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767319178.887769:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319178.887801:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319178.887806:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991644928:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767319178.887812:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319178.887817:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319178.887820:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991645056:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319178.888044:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319178.888050:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767319178.888053:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319178.888059:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4352 00000100:00080000:0.0:1767319178.888068:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319178.888075:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767319178.888124:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400380 x1853163991645184/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319178.888149:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319178.888153:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767319178.888154:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319178.888158:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4352 00000100:00080000:0.0:1767319178.888162:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319178.888166:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767319178.888174:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767319178.888186:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400000 x1853163991645312/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767319178.888392:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991645184:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767319178.888499:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991645312:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319179.079302:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319179.143248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319180.104060:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319180.170440:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319181.128093:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319181.191618:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319182.153562:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319182.215249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319183.175725:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319183.239204:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319184.008930:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319184.008940:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319184.008947:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319184.008953:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319184.199783:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319184.265715:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319185.223296:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319185.287213:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319186.247954:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319186.311230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319187.271359:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319187.335321:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319188.295426:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319188.359436:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319189.127230:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319189.127238:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319189.127240:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319189.127243:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319189.319301:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319189.383220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319190.344640:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319190.407212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319191.369447:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319191.431214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319192.393172:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319192.455240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319193.415609:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319193.479239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319194.247382:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319194.247414:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319194.247416:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319194.247420:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319194.439349:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319194.503231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319195.463566:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319195.527216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319196.489189:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319196.551214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319197.513264:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319197.580075:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319198.536243:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319198.599351:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319199.367268:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319199.367279:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319199.367282:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319199.367285:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319199.560289:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319199.625496:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319200.583352:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319200.647579:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319201.613271:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319201.671258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319202.631337:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319202.695460:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319203.463307:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319178/real 1767319178] req@ffff88c5c6400380 x1853163991645184/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319203 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319203.463346:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400380 x1853163991645184/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319203 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319203.463359:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319178/real 1767319178] req@ffff88c5c6400000 x1853163991645312/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319203 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319203.463371:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400000 x1853163991645312/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319203 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319203.463391:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319203.463421:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319203.463426:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991645184:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319203.463432:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319203.463438:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319203.463441:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991645312:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767319203.464143:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319203.464151:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767319203.464156:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319203.464163:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4378 00000100:00080000:1.0:1767319203.464173:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319203.464179:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767319203.464221:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cfb80 x1853163991645440/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767319203.464255:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319203.464259:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767319203.464260:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319203.464264:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4378 00000100:00080000:1.0:1767319203.464268:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319203.464271:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767319203.464281:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767319203.464293:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ce300 x1853163991645568/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319203.464301:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991645440:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767319203.464412:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991645568:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319203.656239:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319203.719562:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319204.679242:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319204.744867:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319205.703351:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319205.767222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319206.728384:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319206.791221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319207.751498:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319207.815219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319208.583428:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319208.583454:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319208.583457:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319208.583460:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319208.775514:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319208.839231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319209.801732:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319209.863301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319210.823630:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319210.887224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319211.847499:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319211.911508:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319212.871268:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319212.935288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319213.703430:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319213.703441:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319213.703444:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319213.703447:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319213.895321:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319213.959217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319214.919281:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319214.983224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319215.946917:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319216.007975:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319216.967695:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319217.031336:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319217.991921:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319218.057535:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319218.826653:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319218.826678:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319218.826681:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319218.826684:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319219.016542:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319219.079510:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319220.039277:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319220.103233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319221.063887:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319221.127394:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319222.091284:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319222.151405:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319223.114202:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319223.175243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319223.943968:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319223.943979:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319223.943982:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319223.943985:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319224.135446:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319224.199247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319225.160623:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319225.223644:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319226.184636:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319226.247187:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319227.207354:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319227.271227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319228.039233:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319203/real 1767319203] req@ffff88c5c65ce300 x1853163991645568/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319228 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319228.039275:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ce300 x1853163991645568/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319228 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319228.039288:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319203/real 1767319203] req@ffff88c5c65cfb80 x1853163991645440/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319228 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319228.039300:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cfb80 x1853163991645440/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319228 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319228.039318:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319228.039341:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319228.039346:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991645568:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319228.039352:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319228.039357:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319228.039361:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991645440:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319228.039604:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319228.039610:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319228.039613:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319228.039618:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4402 00000100:00080000:2.0:1767319228.039641:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319228.039648:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319228.039692:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5880 x1853163991645696/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767319228.039716:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319228.039719:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319228.039721:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319228.039724:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4402 00000100:00080000:2.0:1767319228.039728:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319228.039731:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319228.039739:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319228.039749:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4380 x1853163991645824/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319228.039812:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991645696:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767319228.039899:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991645824:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319228.231353:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319228.296461:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319229.256124:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319229.319284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319230.284162:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319230.348846:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319231.304133:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319231.367613:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319232.328903:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319232.391197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319233.159832:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319233.159848:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319233.159851:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319233.159856:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319233.351461:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319233.415687:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319234.377447:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319234.439637:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319235.399531:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319235.463856:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319236.423498:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319236.487612:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319237.447365:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319237.511225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319238.280967:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319238.281014:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319238.281017:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319238.281021:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319238.472312:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319238.535330:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319239.495593:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319239.559223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319240.520291:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319240.583197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319241.543510:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319241.607304:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319242.567432:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319242.631221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319243.399578:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319243.399589:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319243.399592:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319243.399595:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319243.591967:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319243.655972:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319244.615395:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319244.679296:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319245.639806:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319245.704618:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319246.664341:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319246.729628:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319247.687613:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319247.751278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319248.524668:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319248.524680:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319248.524682:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319248.524686:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319248.716952:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319248.775224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319249.735396:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319249.799202:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319250.760541:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319250.823288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319251.783325:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319251.847281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319252.810346:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319252.871211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319253.639175:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319253.639186:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319253.639189:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319253.639192:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319253.639677:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319228/real 1767319228] req@ffff88c5c57d5880 x1853163991645696/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319253 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319253.639708:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5880 x1853163991645696/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319253 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319253.639719:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319228/real 1767319228] req@ffff88c5c57d4380 x1853163991645824/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319253 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319253.639730:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4380 x1853163991645824/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319253 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319253.639748:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319253.639768:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319253.639772:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991645696:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319253.639779:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319253.639783:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319253.639787:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991645824:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767319253.639794:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319253.639798:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319253.639801:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319253.639806:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4427 00000100:00080000:2.0:1767319253.639815:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319253.639821:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319253.639848:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5c00 x1853163991645952/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767319253.639873:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319253.639876:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319253.639878:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319253.639882:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4427 00000100:00080000:2.0:1767319253.639885:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319253.639889:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319253.639898:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319253.639909:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4a80 x1853163991646080/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319253.639910:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991645952:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767319253.640010:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991646080:192.168.204.145@tcp:38:kworker.0 00010000:00010000:2.0:1767319253.832422:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319253.895295:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319254.857483:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319254.919236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319255.879671:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319255.943263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319256.903252:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319256.967194:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319257.927391:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319257.991226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319258.760340:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319258.760377:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319258.760381:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319258.760384:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319258.951491:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319259.015251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319259.976220:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319260.039207:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319261.003791:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319261.063365:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319262.028257:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319262.087602:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319263.048941:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319263.111258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319263.880876:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319263.880887:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319263.880890:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319263.880894:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319264.071426:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319264.135209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319265.098833:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319265.159680:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319266.120274:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319266.183985:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319267.143539:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319267.207232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319268.167359:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319268.234758:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319268.999432:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319268.999444:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319268.999447:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319268.999450:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319269.191585:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319269.256477:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319270.220402:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319270.281890:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319271.239721:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319271.303185:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319272.263447:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319272.327226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319273.287446:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319273.351235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319274.119215:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319274.119226:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319274.119229:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319274.119232:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319274.311610:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319274.375585:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319275.335399:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319275.399195:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319276.359907:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319276.423216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319277.383594:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319277.447221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319278.215241:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319253/real 1767319253] req@ffff88c5c57d4a80 x1853163991646080/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319278 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319278.215283:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4a80 x1853163991646080/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319278 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319278.215295:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319253/real 1767319253] req@ffff88c5c57d5c00 x1853163991645952/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319278 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319278.215307:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5c00 x1853163991645952/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319278 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319278.215326:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319278.215356:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319278.215361:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991646080:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319278.215386:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319278.215394:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319278.215399:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991645952:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319278.215412:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319278.215419:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319278.215422:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319278.215431:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4452 00000100:00080000:2.0:1767319278.215441:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319278.215448:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319278.215480:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5f80 x1853163991646208/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767319278.215499:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319278.215501:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319278.215503:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319278.215505:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4452 00000100:00080000:2.0:1767319278.215507:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319278.215510:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319278.215517:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319278.215526:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7480 x1853163991646336/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319278.215765:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991646208:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767319278.215856:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991646336:192.168.204.145@tcp:38:kworker.0 00010000:00010000:2.0:1767319278.407713:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319278.471328:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319279.431328:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319279.495193:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319280.455384:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319280.519303:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319281.482873:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319281.545148:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319282.504894:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319282.567182:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319283.335348:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319283.335358:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319283.335360:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319283.335363:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319283.527551:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319283.591444:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319284.551382:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319284.615452:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319285.575908:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319285.639190:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319286.604725:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319286.668384:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319287.623374:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319287.687252:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319288.455284:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319288.455295:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319288.455298:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319288.455301:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319288.647540:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319288.711474:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319289.671759:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319289.735210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319290.696206:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319290.759698:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319291.721007:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319291.783993:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319292.743483:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319292.807537:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319293.575410:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319293.575419:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319293.575430:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319293.575434:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319293.767364:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319293.831335:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319294.791665:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319294.855726:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319295.815365:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319295.882542:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319296.839387:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319296.903221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319297.863386:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319297.927205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319298.695219:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319298.695230:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319298.695233:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319298.695236:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319298.888542:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319298.951458:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319299.911934:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319299.976898:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319300.935373:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319300.999399:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319301.960089:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319302.023211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319302.984295:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319303.048545:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319303.815231:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319278/real 1767319278] req@ffff88c5c57d5f80 x1853163991646208/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319303 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319303.815244:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319303.815254:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319303.815261:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319303.815266:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767319303.815286:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5f80 x1853163991646208/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319303 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319303.815298:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319278/real 1767319278] req@ffff88c5c57d7480 x1853163991646336/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319303 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319303.815310:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7480 x1853163991646336/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319303 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767319303.815328:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319303.815349:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319303.815370:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991646208:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767319303.815377:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319303.815383:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319303.815386:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991646336:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767319303.815419:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319303.815423:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319303.815425:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319303.815430:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4477 00000100:00080000:2.0:1767319303.815437:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319303.815442:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319303.815467:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6a00 x1853163991646464/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767319303.815488:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319303.815490:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319303.815491:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319303.815494:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4477 00000100:00080000:2.0:1767319303.815496:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319303.815498:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319303.815505:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319303.815512:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4000 x1853163991646592/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767319303.815559:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991646464:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767319303.815645:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991646592:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319304.008280:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319304.073531:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319305.034296:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319305.095243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319306.055442:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319306.119256:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319307.079397:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319307.143473:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319308.103488:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319308.167278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319308.935390:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319308.935400:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319308.935402:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319308.935406:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319309.127635:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319309.192030:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319310.153749:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319310.217435:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319311.176705:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319311.239725:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319312.199669:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319312.263205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319313.224173:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319313.287303:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319314.055337:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319314.055347:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319314.055351:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319314.055355:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319314.247316:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319314.311248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319315.274545:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319315.335202:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319316.295368:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319316.360015:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319317.319298:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319317.384376:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319318.344607:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319318.407955:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319319.182052:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319319.182061:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319319.182063:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319319.182066:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319319.367428:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319319.431230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319320.391498:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319320.455220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319321.417175:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319321.479289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319322.439959:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319322.503226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319323.465396:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319323.527189:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319324.295416:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319324.295425:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319324.295428:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319324.295431:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319324.487459:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319324.551227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319325.511365:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319325.575236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319326.535501:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319326.599231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319327.559500:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319327.623214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319328.391800:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319303/real 1767319303] req@ffff88c5c57d6a00 x1853163991646464/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319328 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319328.391856:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6a00 x1853163991646464/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319328 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319328.391868:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319303/real 1767319303] req@ffff88c5c57d4000 x1853163991646592/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319328 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319328.391880:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4000 x1853163991646592/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319328 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319328.391899:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319328.391992:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319328.391997:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991646464:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319328.392004:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319328.394481:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319328.394492:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767319328.394497:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319328.394506:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4503 00000100:00080000:0.0:1767319328.394517:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319328.394582:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767319328.394618:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400000 x1853163991646720/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319328.394638:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319328.394641:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767319328.394642:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319328.394647:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4503 00000100:00080000:0.0:1767319328.394650:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319328.394654:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767319328.394663:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767319328.394680:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400380 x1853163991646848/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319328.396824:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319328.396835:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991646592:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767319328.396880:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991646720:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767319328.397354:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991646848:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319328.583510:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319328.648567:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319329.610330:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319329.671212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319330.631506:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319330.695253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319331.655389:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319331.719217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319332.679839:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319332.743966:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319333.511493:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319333.511504:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319333.511506:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319333.511510:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319333.704944:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319333.767214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319334.729284:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319334.791575:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319335.751405:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319335.815234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319336.776129:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319336.839234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319337.799456:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319337.863219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319338.634333:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319338.634348:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319338.634352:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319338.634355:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319338.824758:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319338.887247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319339.847545:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319339.911541:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319340.871668:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319340.935509:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319341.896703:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319341.959237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319342.919624:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319342.984167:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319343.751486:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319343.751548:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319343.751551:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319343.751554:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319343.945193:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319344.008977:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319344.967405:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319345.031210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319345.991434:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319346.055449:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319347.015393:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319347.079382:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319348.039485:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319348.103220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319348.872443:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319348.872451:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319348.872454:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319348.872456:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319349.063278:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319349.128709:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319350.089024:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319350.151273:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319351.115229:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319351.175216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319352.135562:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319352.199425:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319353.159226:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319353.223220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319353.992802:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319353.992813:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319353.992816:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319328/real 1767319328] req@ffff88c5c6400000 x1853163991646720/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319353 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319353.992817:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319353.992820:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319353.992851:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400000 x1853163991646720/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319353 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319353.992861:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319328/real 1767319328] req@ffff88c5c6400380 x1853163991646848/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319353 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319353.992873:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400380 x1853163991646848/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319353 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319353.992921:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319353.992942:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319353.992946:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991646720:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319353.992953:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319353.992958:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00080000:0.0:1767319353.992962:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1767319353.992962:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991646848:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319353.992966:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767319353.992970:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319353.992976:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4527 00000100:00080000:0.0:1767319353.992986:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319353.992993:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767319353.993283:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402d80 x1853163991646976/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319353.993442:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991646976:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319353.993736:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319353.993741:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767319353.993743:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319353.993748:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4527 00000100:00080000:0.0:1767319353.993752:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319353.993757:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767319353.993767:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767319353.993791:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6401c00 x1853163991647104/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319353.998450:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991647104:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319354.184132:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319354.247225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319355.207594:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319355.273452:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319356.231849:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319356.295328:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319357.255358:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319357.319279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319358.279527:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319358.343779:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319359.111441:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319359.111451:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319359.111454:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319359.111457:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319359.309163:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319359.367234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319360.328479:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319360.391868:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319361.351438:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319361.415298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319362.375463:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319362.439448:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319363.399512:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319363.463292:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319364.231252:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319364.231297:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319364.231301:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319364.231305:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319364.424005:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319364.487368:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319365.449634:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319365.511293:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319366.474610:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319366.535655:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319367.495516:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319367.559305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319368.519692:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319368.584487:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319369.351264:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319369.351278:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319369.351281:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319369.351285:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319369.543599:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319369.608369:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319370.567721:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319370.632287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319371.591670:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319371.655273:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319372.619029:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319372.680536:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319373.639380:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319373.703363:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319374.471429:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319374.471441:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319374.471444:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319374.471447:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319374.663894:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319374.727381:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319375.687410:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319375.751573:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319376.711811:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319376.775284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319377.736140:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319377.799183:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319378.567220:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319353/real 1767319354] req@ffff88c5c6401c00 x1853163991647104/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319378 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319378.567273:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6401c00 x1853163991647104/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319378 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319378.567284:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319353/real 1767319353] req@ffff88c5c6402d80 x1853163991646976/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319378 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319378.567295:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402d80 x1853163991646976/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319378 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319378.567312:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319378.567333:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319378.567338:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6401c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991647104:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319378.567343:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319378.567349:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319378.567353:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991646976:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319378.567365:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319378.567371:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319378.567374:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319378.567381:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4553 00000100:00080000:2.0:1767319378.567389:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319378.567402:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319378.567428:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6300 x1853163991647232/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319378.567630:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991647232:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319378.567720:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319378.567724:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319378.567725:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319378.567729:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4553 00000100:00080000:2.0:1767319378.567733:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319378.567736:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319378.567744:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319378.567756:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7800 x1853163991647360/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767319378.567944:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991647360:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319378.759693:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319378.825372:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319379.784587:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319379.847677:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319380.809402:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319380.871779:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319381.831650:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319381.896179:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319382.855329:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319382.919212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319383.689866:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319383.689881:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319383.689885:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319383.689889:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319383.879922:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319383.943249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319384.903702:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319384.967248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319385.927659:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319385.991534:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319386.955691:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319387.015278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319387.975374:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319388.039243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319388.807210:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319388.807221:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319388.807224:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319388.807227:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319388.999573:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319389.063541:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319390.023918:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319390.087239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319391.050830:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319391.111237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319392.072690:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319392.135288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319393.095438:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319393.159217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319393.927316:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319393.927326:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319393.927329:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319393.927332:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319394.120225:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319394.183220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319395.143319:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319395.209072:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319396.167286:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319396.231323:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319397.191403:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319397.256931:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319398.215564:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319398.279607:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319399.047608:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319399.047635:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319399.047637:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319399.047641:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319399.241086:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319399.303191:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319400.263319:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319400.327231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319401.288250:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319401.351681:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319402.311502:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319402.375175:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319403.143226:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319378/real 1767319378] req@ffff88c5c57d7800 x1853163991647360/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319403 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319403.143262:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7800 x1853163991647360/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319403 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319403.143274:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319378/real 1767319378] req@ffff88c5c57d6300 x1853163991647232/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319403 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319403.143286:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6300 x1853163991647232/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319403 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319403.143330:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319403.143366:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319403.143371:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991647360:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767319403.143377:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319403.143382:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319403.143385:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991647232:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319403.143463:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319403.143469:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319403.143473:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319403.143479:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4577 00000100:00080000:3.0:1767319403.143488:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319403.143496:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319403.143526:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728a80 x1853163991647488/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319403.143549:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319403.143552:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319403.143554:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319403.143557:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4577 00000100:00080000:3.0:1767319403.143560:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319403.143564:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319403.143572:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319403.143591:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b480 x1853163991647616/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767319403.143669:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991647488:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767319403.143829:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991647616:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319403.335452:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319403.400711:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319404.359672:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319404.425572:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319405.383401:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319405.447250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319406.408448:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319406.471218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319407.431442:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319407.495560:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319408.266554:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319408.266565:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319408.266568:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319408.266571:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319408.459528:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319408.520700:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319409.479473:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319409.544956:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319410.505338:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319410.568889:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319411.527353:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319411.591436:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319412.552118:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319412.615426:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319413.383213:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319413.383234:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319413.383238:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319413.383257:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319413.575487:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319413.641277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319414.606703:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319414.663345:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319415.625567:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319415.689383:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319416.648272:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319416.711227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319417.671653:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319417.735246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319418.503459:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319418.503478:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319418.503482:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319418.503485:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319418.696305:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319418.759322:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319419.719318:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319419.785297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319420.743337:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319420.808017:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319421.768134:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319421.831817:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319422.791488:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319422.855244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319423.623344:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319423.623363:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319423.623367:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319423.623370:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319423.815365:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319423.879514:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319424.839298:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319424.903245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319425.864864:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319425.927823:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319426.890777:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319426.951261:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319427.912049:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319427.975678:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319428.743228:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319403/real 1767319403] req@ffff88c5c5728a80 x1853163991647488/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319428 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319428.743272:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728a80 x1853163991647488/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319428 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319428.743283:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319403/real 1767319403] req@ffff88c5c572b480 x1853163991647616/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319428 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319428.743295:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b480 x1853163991647616/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319428 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319428.743334:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319428.743343:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319428.743347:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991647488:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319428.743364:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319428.743370:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319428.743374:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991647616:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319428.743375:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319428.743382:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319428.743386:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319428.743393:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4602 00000100:00080000:3.0:1767319428.743403:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319428.743409:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319428.743439:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572a680 x1853163991647744/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319428.743470:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319428.743473:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319428.743475:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319428.743479:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4602 00000100:00080000:3.0:1767319428.743483:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319428.743487:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319428.743496:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319428.743508:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572bb80 x1853163991647872/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767319428.743587:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991647744:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767319428.743675:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991647872:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319428.935496:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319429.000002:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319429.960428:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319430.024826:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319430.983331:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319431.047205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319432.007360:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319432.071201:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319433.033419:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319433.095235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319433.863396:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319433.863408:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319433.863411:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319433.863414:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319434.055647:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319434.120182:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319435.079354:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319435.144081:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319436.103470:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319436.167232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319437.127284:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319437.191377:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319438.151855:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319438.215425:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319438.983318:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319438.983330:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319438.983333:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319438.983336:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319439.175557:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319439.239353:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319440.199377:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319440.264566:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319441.224032:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319441.287412:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319442.247806:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319442.311980:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319443.271331:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319443.335225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319444.104745:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319444.104756:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319444.104760:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319444.104767:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319444.295312:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319444.359210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319445.319322:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319445.383219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319446.343311:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319446.407216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319447.367311:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319447.431896:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319448.391445:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319448.455454:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319449.230140:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319449.230151:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319449.230154:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319449.230157:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319449.415781:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319449.479239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319450.439568:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319450.503274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319451.463892:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319451.527536:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319452.487453:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319452.551265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319453.319760:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319428/real 1767319428] req@ffff88c5c572a680 x1853163991647744/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319453 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319453.319794:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572a680 x1853163991647744/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319453 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319453.319805:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319428/real 1767319428] req@ffff88c5c572bb80 x1853163991647872/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319453 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319453.319816:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572bb80 x1853163991647872/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319453 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319453.319835:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319453.319880:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319453.319885:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991647744:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319453.319892:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319453.319898:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319453.319901:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991647872:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767319453.319935:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319453.319942:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767319453.319953:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319453.319958:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4627 00000100:00080000:1.0:1767319453.319967:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319453.319973:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767319453.320001:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cea00 x1853163991648000/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767319453.320026:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319453.320029:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767319453.320031:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319453.320035:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4627 00000100:00080000:1.0:1767319453.320038:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319453.320042:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767319453.320050:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767319453.320062:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cfb80 x1853163991648128/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767319453.320216:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991648000:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767319453.320364:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991648128:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319453.511299:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319453.575859:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319454.535500:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319454.599210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319455.559434:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319455.623395:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319456.586904:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319456.647486:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319457.607322:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319457.671197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319458.439335:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319458.439345:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319458.439348:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319458.439351:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319458.631421:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319458.695913:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319459.655917:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319459.719521:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319460.679795:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319460.743259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319461.705895:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319461.771047:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319462.727299:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319462.791233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319463.559567:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319463.559578:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319463.559596:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319463.559599:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319463.751346:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319463.815210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319464.775713:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319464.839606:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319465.801349:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319465.863242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319466.827050:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319466.887220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319467.847294:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319467.911302:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319468.679253:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319468.679266:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319468.679269:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319468.679271:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319468.874649:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319468.935614:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319469.898150:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319469.959778:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319470.919518:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319470.983261:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319471.944361:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319472.007222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319472.967351:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319473.032367:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319473.799299:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319473.799311:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319473.799314:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319473.799317:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319473.991411:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319474.055721:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319475.018150:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319475.079212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319476.039293:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319476.103272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319477.063250:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319477.127201:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319478.088042:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319478.152654:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319478.919231:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319478.919241:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319478.919244:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319478.919247:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319478.920580:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319453/real 1767319453] req@ffff88c5c65cea00 x1853163991648000/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319478 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319478.920616:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cea00 x1853163991648000/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319478 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319478.920626:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319453/real 1767319453] req@ffff88c5c65cfb80 x1853163991648128/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319478 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319478.920636:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cfb80 x1853163991648128/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319478 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319478.920677:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319478.920703:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319478.920708:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991648000:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319478.920714:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319478.920720:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319478.920723:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991648128:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767319478.920817:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319478.920823:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319478.920826:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319478.920832:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4652 00000100:00080000:2.0:1767319478.920840:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319478.920846:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319478.920885:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6300 x1853163991648256/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319478.920942:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991648256:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319478.921067:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319478.921071:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319478.921073:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319478.921077:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4652 00000100:00080000:2.0:1767319478.921082:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319478.921085:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319478.921112:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319478.921127:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7800 x1853163991648384/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319478.921300:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991648384:192.168.204.145@tcp:38:kworker.0 00010000:00010000:2.0:1767319479.111689:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319479.175290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319480.135309:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319480.202655:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319481.161091:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319481.223229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319482.183552:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319482.247542:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319483.207430:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319483.271298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319484.039583:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319484.039596:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319484.039599:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319484.039603:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319484.232802:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319484.295278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319485.257603:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319485.319660:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319486.281030:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319486.343262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319487.303510:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319487.368786:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319488.327324:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319488.391554:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319489.161599:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319489.161614:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319489.161618:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319489.161621:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319489.351439:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319489.415940:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319490.375713:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319490.440824:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319491.399490:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319491.463269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319492.423418:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319492.487562:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319493.447720:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319493.511525:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319494.279506:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319494.279535:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319494.279540:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319494.279544:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319494.473489:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319494.537634:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319495.495419:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319495.559347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319496.519395:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319496.583327:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319497.544488:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319497.607470:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319498.568615:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319498.631358:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319499.399438:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319499.399449:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319499.399452:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319499.399455:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319499.591959:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319499.655790:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319500.615252:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319500.679209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319501.643454:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319501.703249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319502.663959:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319502.728241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319503.495231:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319478/real 1767319478] req@ffff88c5c57d7800 x1853163991648384/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319503 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319503.495266:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7800 x1853163991648384/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319503 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319503.495276:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319478/real 1767319478] req@ffff88c5c57d6300 x1853163991648256/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319503 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767319503.495287:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6300 x1853163991648256/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319503 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767319503.495304:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319503.495341:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319503.495346:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991648384:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319503.495351:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319503.495356:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767319503.495359:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991648256:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319503.495481:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319503.495486:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319503.495489:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319503.495495:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4678 00000100:00080000:2.0:1767319503.495504:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319503.495510:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319503.495538:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4e00 x1853163991648512/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767319503.495579:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319503.495582:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319503.495584:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319503.495588:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4678 00000100:00080000:2.0:1767319503.495591:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319503.495594:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319503.495603:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319503.495616:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5500 x1853163991648640/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767319503.495761:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991648512:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767319503.495883:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991648640:192.168.204.145@tcp:38:kworker.0 00010000:00010000:2.0:1767319503.691825:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319503.751798:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319504.712352:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319504.775239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319505.735364:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319505.800802:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319506.759327:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319506.823899:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319507.783410:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319507.847455:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319508.619628:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319508.619639:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319508.619642:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319508.619645:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319508.845551:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319508.871900:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319509.895241:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319509.895405:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319510.919277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319510.919597:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319511.945129:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319511.945353:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319512.969018:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319512.969395:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319513.739543:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319513.739555:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319513.739558:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319513.739561:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319513.991218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319513.991437:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319515.015225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319515.016656:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319516.039280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319516.039491:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319517.063203:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319517.065344:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319518.087223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319518.088685:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767319518.855497:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319518.855512:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319518.855516:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319518.855520:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319519.112857:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319519.113065:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319520.135264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319520.135416:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319521.159451:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319521.159463:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319522.183291:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319522.184269:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319523.207219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319523.209281:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319523.975473:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319523.975484:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319523.975487:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319523.975491:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319524.231233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319524.231890:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319525.255870:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319525.255950:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319526.279287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319526.279811:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319527.305621:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319527.306071:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319528.072406:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319503/real 1767319503] req@ffff88c5c57d4e00 x1853163991648512/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319528 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319528.072470:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4e00 x1853163991648512/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319528 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319528.072483:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319503/real 1767319503] req@ffff88c5c57d5500 x1853163991648640/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319528 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319528.072496:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5500 x1853163991648640/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319528 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319528.072524:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319528.072549:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319528.072554:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991648512:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319528.072561:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319528.072566:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319528.072570:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991648640:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319528.072790:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319528.072796:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767319528.072799:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319528.072805:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4702 00000100:00080000:0.0:1767319528.072813:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319528.072820:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767319528.072852:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403100 x1853163991648768/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319528.072901:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319528.072905:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767319528.072907:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319528.072911:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4702 00000100:00080000:0.0:1767319528.072916:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319528.072920:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767319528.072930:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767319528.072943:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403b80 x1853163991648896/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767319528.073871:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991648768:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767319528.073981:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991648896:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767319528.330406:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319528.330616:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319529.351228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319529.352231:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319530.375217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319530.381627:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319531.399256:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319531.399393:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319532.423220:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319532.427539:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319533.191265:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319533.191276:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319533.191279:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319533.191282:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319533.447204:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319533.448110:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319534.471280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319534.471841:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319535.495642:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319535.497048:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319536.519550:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319536.519798:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319537.543212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319537.543920:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319538.315995:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319538.316020:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319538.316022:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319538.316025:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319538.567564:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319538.567957:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319539.591543:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319539.592420:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319540.615667:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319540.615875:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319541.639327:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319541.639813:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319542.663347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319542.664048:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319543.431226:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319543.431251:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319543.431255:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319543.431260:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319543.687360:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319543.689376:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319544.712262:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319544.714087:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319545.735506:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319545.736868:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319546.759403:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319546.760219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319547.783323:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319547.783802:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319548.551461:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319548.551474:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319548.551478:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319548.551482:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319548.807489:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319548.807529:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319549.831219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319549.831570:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319550.855787:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319550.860176:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319551.879312:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319551.879464:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319552.903428:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319552.903593:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319553.673913:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319553.673922:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319553.673925:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319553.673929:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767319553.674059:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319528/real 1767319528] req@ffff88c5c6403100 x1853163991648768/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319553 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319553.674128:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403100 x1853163991648768/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319553 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319553.674136:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319528/real 1767319528] req@ffff88c5c6403b80 x1853163991648896/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319553 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319553.674146:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403b80 x1853163991648896/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319553 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319553.674164:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319553.674182:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319553.674186:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991648768:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319553.674193:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319553.674197:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319553.674200:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991648896:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767319553.674719:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319553.674725:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767319553.674728:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319553.674741:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4727 00000100:00080000:2.0:1767319553.674751:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319553.674757:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767319553.674789:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5500 x1853163991649024/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767319553.674815:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319553.674818:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767319553.674819:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767319553.674823:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4727 00000100:00080000:2.0:1767319553.674827:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767319553.674830:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767319553.674838:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767319553.674850:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4e00 x1853163991649152/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319553.674987:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991649024:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319553.675082:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991649152:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767319553.927270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319553.927511:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319554.956298:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319554.956627:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319555.975293:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319555.975445:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319556.999448:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319556.999629:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319558.023255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319558.023873:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319558.793696:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319558.793707:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319558.793710:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319558.793713:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319559.047204:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319559.047387:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319560.071426:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319560.071698:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319561.095686:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319561.096926:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319562.119354:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319562.119531:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319563.143722:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319563.144234:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767319563.912855:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319563.912866:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319563.912869:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319563.912872:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319564.167322:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319564.167437:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319565.191225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319565.196044:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319566.215972:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319566.216071:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319567.239369:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319567.240822:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319568.263585:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319568.264346:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319569.032336:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319569.032351:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319569.032364:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319569.032368:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319569.287428:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319569.287679:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319570.311573:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319570.313033:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319571.335223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319571.335282:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319572.359415:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319572.359582:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319573.383317:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319573.383369:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319574.151780:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319574.151795:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319574.151808:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319574.151812:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319574.407451:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319574.407767:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319575.431269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319575.431534:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319576.455318:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319576.455492:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319577.479266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319577.481233:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319578.247398:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319553/real 1767319553] req@ffff88c5c57d5500 x1853163991649024/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319578 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319578.247445:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5500 x1853163991649024/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319578 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319578.247457:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319553/real 1767319553] req@ffff88c5c57d4e00 x1853163991649152/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319578 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319578.247471:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4e00 x1853163991649152/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319578 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319578.247495:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319578.249816:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319578.249831:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991649024:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319578.249844:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319578.249853:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319578.249857:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991649152:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767319578.250105:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319578.250110:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767319578.250135:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319578.250141:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4752 00000100:00080000:1.0:1767319578.250154:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319578.250161:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767319578.250211:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ce300 x1853163991649280/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767319578.250234:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319578.250236:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767319578.250238:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319578.250240:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4752 00000100:00080000:1.0:1767319578.250243:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319578.250246:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767319578.250255:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767319578.250264:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf100 x1853163991649408/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319578.250429:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991649280:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319578.250637:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991649408:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767319578.504630:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319578.508141:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319579.527296:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319579.527446:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319580.551240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319580.552081:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319581.575342:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319581.579412:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319582.599549:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319582.599598:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319583.367252:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319583.367263:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319583.367267:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319583.367271:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767319583.623254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319583.626669:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767319584.647267:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319584.647341:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319585.671218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319585.671282:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319586.695202:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319586.695319:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319587.719218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319587.719491:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319588.491209:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319588.491219:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319588.491222:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319588.491226:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319588.743216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319588.746238:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319589.767270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319589.769419:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319590.791295:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319590.791392:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319591.815242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319591.819258:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767319592.839471:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319592.839573:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319593.607468:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319593.607504:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319593.607507:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319593.607511:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319593.864476:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319593.864586:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319594.887226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319594.887437:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319595.911335:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319595.916879:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319596.935521:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319596.935548:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319597.959323:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319597.959512:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319598.730393:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319598.730409:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319598.730412:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319598.730415:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767319598.984670:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319598.984979:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319600.008118:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319600.008821:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319601.034718:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319601.035203:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319602.055505:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319602.055648:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319603.079385:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319603.080089:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319603.847362:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319603.847376:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319603.847379:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319603.847382:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319603.847483:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319578/real 1767319578] req@ffff88c5c65ce300 x1853163991649280/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319603 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319603.847521:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ce300 x1853163991649280/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319603 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319603.847531:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319578/real 1767319578] req@ffff88c5c65cf100 x1853163991649408/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319603 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319603.847541:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf100 x1853163991649408/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319603 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319603.847562:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319603.847583:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319603.847587:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991649280:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319603.847593:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319603.847598:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319603.847601:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991649408:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319603.847675:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319603.847680:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767319603.847683:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319603.847689:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4777 00000100:00080000:0.0:1767319603.847701:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319603.847709:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767319603.847763:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403b80 x1853163991649536/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319603.847790:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319603.847815:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767319603.847817:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319603.847822:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4777 00000100:00080000:0.0:1767319603.847826:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319603.847830:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767319603.847840:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767319603.847853:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6403100 x1853163991649664/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767319603.847911:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991649536:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767319603.848234:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991649664:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767319604.103401:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319604.104445:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319605.128009:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319605.128143:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319606.151583:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319606.151631:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319607.175669:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319607.177085:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319608.199705:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319608.199783:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319608.967357:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319608.967367:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319608.967370:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319608.967381:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319609.223213:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319609.224757:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319610.247215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319610.247306:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319611.271401:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319611.273166:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319612.295841:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319612.296702:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319613.319245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319613.319411:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319614.087218:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319614.087229:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319614.087232:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319614.087236:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319614.343213:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319614.343347:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319615.367223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319615.367773:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319616.391472:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319616.393661:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319617.415391:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319617.415707:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319618.439217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319618.439664:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319619.207318:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319619.207330:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319619.207334:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319619.207338:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319619.463209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319619.463358:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319620.487467:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319620.488211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319621.511798:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319621.512084:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319622.535239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319622.537428:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319623.559488:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319623.563265:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319624.336034:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319624.336045:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319624.336049:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319624.336052:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767319624.583233:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319624.584074:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319625.607216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319625.607364:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319626.631205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319626.631307:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319627.655231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319627.655605:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319628.423229:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319603/real 1767319603] req@ffff88c5c6403b80 x1853163991649536/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319628 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319628.423274:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403b80 x1853163991649536/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319628 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319628.423285:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319603/real 1767319603] req@ffff88c5c6403100 x1853163991649664/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319628 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319628.423298:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6403100 x1853163991649664/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319628 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319628.423343:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319628.423374:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319628.423379:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991649536:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319628.423386:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319628.423393:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319628.423396:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6403100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991649664:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319628.423495:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319628.423501:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319628.423504:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319628.423508:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4803 00000100:00080000:3.0:1767319628.423522:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319628.423528:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319628.423553:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572bb80 x1853163991649792/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767319628.423610:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991649792:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319628.423721:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319628.423724:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319628.423726:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319628.423731:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4803 00000100:00080000:3.0:1767319628.423735:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319628.423739:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319628.423747:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319628.423760:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572a680 x1853163991649920/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767319628.424027:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991649920:192.168.204.145@tcp:38:kworker.0 00010000:00010000:2.0:1767319628.679639:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319628.680080:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319629.703263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319629.704035:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319630.727665:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319630.728057:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319631.751299:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319631.751352:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319632.775263:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319632.775735:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319633.543240:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319633.543249:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319633.543252:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319633.543256:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319633.799514:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319633.799599:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319634.823977:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319634.825701:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319635.847494:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319635.847825:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319636.871837:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319636.874426:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319637.896370:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319637.898251:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319638.663423:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319638.663433:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319638.663451:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319638.663455:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319638.919330:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319638.921772:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319639.943217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319639.943374:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319640.967223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319640.967393:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319641.991209:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319641.991348:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319643.015353:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319643.015439:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319643.783207:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319643.783218:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319643.783221:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319643.783225:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319644.039404:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319644.040481:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319645.063471:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319645.064277:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319646.087217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319646.087457:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319647.111237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319647.111885:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319648.135475:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319648.135820:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319648.903241:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319648.903251:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319648.903254:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319648.903257:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767319649.160057:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319649.160275:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319650.183225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319650.188582:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319651.207468:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319651.207749:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319652.231352:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319652.232860:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319653.002557:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319628/real 1767319628] req@ffff88c5c572a680 x1853163991649920/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319653 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319653.002614:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572a680 x1853163991649920/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319653 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319653.002627:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319628/real 1767319628] req@ffff88c5c572bb80 x1853163991649792/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319653 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319653.002641:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572bb80 x1853163991649792/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319653 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319653.002702:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319653.002737:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319653.002742:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991649920:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319653.002750:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319653.002757:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319653.002760:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991649792:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767319653.002863:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319653.002876:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319653.002879:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319653.002885:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4827 00000100:00080000:3.0:1767319653.002897:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319653.002905:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319653.002949:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b480 x1853163991650048/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319653.002981:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319653.002984:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319653.002987:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319653.002991:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4827 00000100:00080000:3.0:1767319653.002995:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319653.002999:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319653.003012:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319653.003026:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728a80 x1853163991650176/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319653.003277:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991650048:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319653.003412:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991650176:192.168.204.145@tcp:38:kworker.0 00000100:00100000:1.0:1767319653.255513:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319653.255798:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319654.279232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319654.279338:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319655.303226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319655.303879:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319656.328140:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319656.329602:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319657.351298:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319657.351301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319658.119347:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319658.119593:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319658.119596:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319658.119598:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319658.375268:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319658.375449:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319659.399349:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319659.399626:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319660.423874:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319660.424164:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319661.447258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319661.447478:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319662.471357:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319662.471493:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319663.239393:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319663.239404:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319663.239407:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319663.239410:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319663.495223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319663.496759:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319664.519267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319664.519463:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319665.544547:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319665.544641:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319666.567364:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319666.567388:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319667.592012:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319667.596830:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319668.360481:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319668.360495:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319668.360498:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319668.360502:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767319668.615255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319668.615460:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319669.639778:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319669.641472:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319670.663535:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319670.663537:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319671.687313:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319671.687451:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319672.711292:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319672.711533:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319673.479357:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319673.479368:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319673.479370:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319673.479373:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319673.735301:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319673.744567:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319674.759347:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319674.759438:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319675.783628:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319675.784812:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319676.807214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319676.807543:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319677.831250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319677.831319:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319678.599453:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319653/real 1767319653] req@ffff88c5c572b480 x1853163991650048/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319678 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319678.599488:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b480 x1853163991650048/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319678 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319678.599500:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319653/real 1767319653] req@ffff88c5c5728a80 x1853163991650176/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319678 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319678.599512:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728a80 x1853163991650176/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319678 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319678.599530:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319678.599568:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319678.599573:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991650048:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319678.599580:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319678.599585:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319678.599604:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991650176:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767319678.599951:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319678.599956:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767319678.599960:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319678.599965:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4852 00000100:00080000:1.0:1767319678.599974:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319678.599980:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767319678.600009:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd880 x1853163991650304/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767319678.600032:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319678.600042:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767319678.600044:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319678.600048:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4852 00000100:00080000:1.0:1767319678.600051:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319678.600054:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767319678.600071:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767319678.600083:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf480 x1853163991650432/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319678.600614:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991650304:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319678.600706:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991650432:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767319678.855221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319678.855547:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319679.879227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319679.880150:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319680.903449:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319680.907386:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319681.927274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319681.929594:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319682.951223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319682.951229:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319683.720057:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319683.720068:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319683.720071:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319683.720075:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319683.975201:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319683.977537:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319684.999284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319684.999503:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319686.023199:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319686.023299:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319687.047284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319687.047407:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319688.071234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319688.072695:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319688.840256:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319688.840267:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319688.840344:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319688.840349:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319689.095419:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319689.095495:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319690.121246:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319690.121526:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319691.143275:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319691.144876:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319692.167570:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319692.171495:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319693.191227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319693.192121:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319693.959233:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319693.959244:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319693.959247:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319693.959257:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319694.215357:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319694.215509:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767319695.239331:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319695.239436:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319696.263345:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319696.265855:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319697.287298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319697.291378:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319698.311269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319698.311282:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319699.079381:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319699.079390:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319699.079398:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319699.079402:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:0.0:1767319699.335311:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319699.335415:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319700.359297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319700.359470:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319701.383282:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319701.387827:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319702.407264:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319702.407375:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319703.175305:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319678/real 1767319678] req@ffff88c5c65cd880 x1853163991650304/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319703 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319703.175359:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd880 x1853163991650304/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319703 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319703.175373:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319678/real 1767319678] req@ffff88c5c65cf480 x1853163991650432/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319703 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319703.175417:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf480 x1853163991650432/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319703 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319703.175442:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319703.175478:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319703.175483:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991650304:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319703.175491:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319703.175506:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319703.175510:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991650432:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767319703.176456:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319703.176465:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767319703.176470:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319703.176478:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4877 00000100:00080000:1.0:1767319703.176492:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319703.176511:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767319703.176559:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cce00 x1853163991650560/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767319703.176590:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319703.176594:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767319703.176596:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319703.176615:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4877 00000100:00080000:1.0:1767319703.176620:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319703.176624:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767319703.176636:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767319703.176649:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cc380 x1853163991650688/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319703.176990:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991650560:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319703.177155:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991650688:192.168.204.145@tcp:38:kworker.0 00010000:00010000:0.0:1767319703.431320:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319703.431817:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319704.455389:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319704.455661:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319705.479591:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319705.480214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319706.503432:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319706.503891:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767319707.527311:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319707.527435:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319708.296182:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319708.296197:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319708.296200:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319708.296204:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319708.551370:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319708.551371:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319709.575426:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319709.577560:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319710.599355:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319710.599449:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767319711.623279:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319711.623318:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319712.647269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319712.649615:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319713.415551:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319713.415565:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319713.415569:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319713.415573:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767319713.671322:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319713.671396:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319714.695248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319714.696144:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767319715.719351:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319715.719467:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319716.743223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319716.743514:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319717.775812:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319717.780919:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319718.538281:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319718.538291:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319718.538294:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319718.538297:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767319718.791270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319718.791492:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767319719.815309:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319719.816662:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319720.839301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319720.841525:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319721.863278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319721.863580:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319722.887313:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319722.888303:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319723.657409:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319723.657420:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319723.657443:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319723.657447:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319723.911249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319723.914154:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319724.935943:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319724.937515:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319725.960155:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319725.960242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319726.983344:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319726.984372:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319728.007354:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319728.009718:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319728.777301:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319728.777316:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319728.777345:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319728.777349:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319728.777479:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319703/real 1767319703] req@ffff88c5c65cce00 x1853163991650560/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319728 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319728.777515:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cce00 x1853163991650560/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319728 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319728.777524:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319703/real 1767319703] req@ffff88c5c65cc380 x1853163991650688/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319728 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319728.777535:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cc380 x1853163991650688/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319728 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319728.777558:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319728.777577:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319728.777582:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991650560:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319728.777588:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319728.777601:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319728.777604:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991650688:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319728.777767:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319728.777772:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1767319728.777776:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319728.777782:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4902 00000100:00080000:0.0:1767319728.777793:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319728.777800:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:0.0:1767319728.777837:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6400e00 x1853163991650816/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319728.777862:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319728.777866:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1767319728.777868:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1767319728.777872:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4902 00000100:00080000:0.0:1767319728.777875:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1767319728.777879:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:0.0:1767319728.777889:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1767319728.777901:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c6402680 x1853163991650944/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767319728.777960:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991650816:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767319728.778075:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991650944:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767319729.031642:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319729.032142:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319730.057731:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319730.057765:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319731.079308:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319731.079371:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319732.104764:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319732.105434:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319733.127423:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319733.130313:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319733.895198:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319733.895209:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319733.895213:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319733.895216:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319734.151244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319734.151401:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319735.175225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319735.175443:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319736.199192:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319736.199357:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319737.223212:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319737.223237:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319738.247353:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319738.247694:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319739.015579:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319739.015605:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319739.015608:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319739.015611:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319739.271761:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319739.271832:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319740.296640:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319740.297768:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319741.319240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319741.319763:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319742.343290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319742.343572:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319743.367377:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319743.368079:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767319744.135386:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319744.135397:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319744.135401:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319744.135405:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319744.392563:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319744.392642:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319745.416536:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319745.416539:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319746.439253:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319746.441059:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319747.463203:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319747.463462:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319748.487224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319748.487599:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319749.267013:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319749.267023:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319749.267026:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319749.267030:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319749.511321:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319749.511390:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319750.535323:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319750.536312:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319751.559348:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319751.559921:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319752.583261:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319752.583526:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319753.351244:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319728/real 1767319728] req@ffff88c5c6400e00 x1853163991650816/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319753 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319753.351304:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6400e00 x1853163991650816/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319753 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319753.351316:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319728/real 1767319728] req@ffff88c5c6402680 x1853163991650944/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319753 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319753.351328:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c6402680 x1853163991650944/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319753 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319753.351347:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319753.351377:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319753.351382:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6400e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991650816:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319753.351388:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319753.351394:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319753.351397:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c6402680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991650944:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319753.351501:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319753.351507:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319753.351510:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319753.351515:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4927 00000100:00080000:3.0:1767319753.351525:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319753.351531:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319753.351561:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729880 x1853163991651072/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319753.351587:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319753.351591:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319753.351593:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319753.351597:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4927 00000100:00080000:3.0:1767319753.351600:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319753.351611:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319753.351620:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319753.351633:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728e00 x1853163991651200/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319753.351707:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991651072:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319753.351795:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991651200:192.168.204.145@tcp:38:kworker.0 00000100:00100000:1.0:1767319753.608237:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319753.608928:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319754.631378:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319754.631486:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319755.655373:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319755.655583:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319756.679919:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319756.679987:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319757.703442:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319757.703571:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319758.474407:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319758.474417:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319758.474420:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319758.474423:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767319758.727328:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319758.727598:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319759.751304:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319759.752256:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319760.775159:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319760.775351:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319761.799260:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319761.802025:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319762.823344:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319762.834314:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319763.591517:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319763.591530:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319763.591551:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319763.591555:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767319763.848187:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319763.848881:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:1.0:1767319764.871219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319764.871809:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319765.895313:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319765.895485:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319766.919217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319766.920877:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319767.943225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319767.947725:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767319768.720676:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319768.720696:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319768.720699:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319768.720703:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767319768.967419:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319768.967818:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319769.991231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319769.993225:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319771.015221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319771.015500:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319772.039216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319772.048580:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319773.063222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319773.063313:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319773.833714:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319773.833724:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319773.833727:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319773.833731:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767319774.087378:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319774.087457:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319775.111206:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319775.111279:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319776.135214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319776.135402:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319777.159236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319777.161034:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319778.183236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319778.183385:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319778.951198:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1767319778.951199:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319753/real 1767319753] req@ffff88c5c5729880 x1853163991651072/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319778 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767319778.951210:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319778.951214:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319778.951234:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767319778.951256:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729880 x1853163991651072/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319778 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319778.951268:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319753/real 1767319753] req@ffff88c5c5728e00 x1853163991651200/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319778 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319778.951279:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728e00 x1853163991651200/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319778 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319778.951296:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319778.951317:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319778.951321:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991651072:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319778.951327:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319778.951332:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319778.951335:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991651200:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767319778.951352:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319778.951357:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767319778.951360:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319778.951366:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4952 00000100:00080000:1.0:1767319778.951375:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319778.951382:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767319778.951412:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cca80 x1853163991651328/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319778.951471:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991651328:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767319778.951535:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319778.951540:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767319778.951542:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319778.951547:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4952 00000100:00080000:1.0:1767319778.951552:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319778.951556:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767319778.951566:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767319778.951580:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ced80 x1853163991651456/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319778.951645:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991651456:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319779.207843:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319779.208437:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319780.231271:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319780.231364:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319781.255278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319781.255406:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319782.279490:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319782.280944:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319783.303860:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319783.304571:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767319784.071295:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319784.071308:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319784.071312:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319784.071316:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319784.327286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319784.327679:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319785.354959:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319785.355773:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319786.375334:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319786.375468:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319787.409683:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319787.410556:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319788.423827:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319788.424854:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319789.191453:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319789.191466:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319789.191469:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319789.191472:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319789.447357:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319789.447514:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319790.471336:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319790.472669:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319791.495179:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319791.498063:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319792.519819:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319792.520640:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319793.544673:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319793.544743:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319794.311640:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319794.311651:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319794.311671:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319794.311675:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319794.567404:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319794.569045:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319795.592856:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319795.594856:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319796.615244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319796.615330:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767319797.642425:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319797.642439:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319798.665084:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319798.666283:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767319799.431894:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319799.431906:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319799.431909:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319799.431913:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319799.687410:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319799.687611:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319800.711211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319800.711437:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319801.735505:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319801.735711:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319802.759509:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319802.760023:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319803.527321:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319778/real 1767319778] req@ffff88c5c65ced80 x1853163991651456/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319803 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319803.527387:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ced80 x1853163991651456/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319803 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319803.527398:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319778/real 1767319778] req@ffff88c5c65cca80 x1853163991651328/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319803 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319803.527408:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cca80 x1853163991651328/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319803 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319803.527427:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319803.527455:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319803.527460:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991651456:192.168.204.145@tcp:38:kworker.0 00000100:00080000:0.0:1767319803.527466:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319803.527472:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319803.527475:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991651328:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767319803.527955:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319803.527961:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767319803.527964:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319803.527970:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 4978 00000100:00080000:1.0:1767319803.527980:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319803.527995:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767319803.528055:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cdf80 x1853163991651584/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767319803.528088:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319803.528092:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767319803.528125:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319803.528129:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 4978 00000100:00080000:1.0:1767319803.528133:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319803.528137:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767319803.528148:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767319803.528160:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cc000 x1853163991651712/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319803.528820:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991651584:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319803.528935:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991651712:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767319803.783341:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319803.787166:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319804.809179:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319804.809222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319805.831498:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319805.831631:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319806.855305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319806.855587:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319807.879288:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319807.879293:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319808.647318:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319808.647329:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319808.647332:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319808.647335:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767319808.905266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319808.905821:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767319809.927567:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319809.927652:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319810.953535:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319810.954187:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319811.976711:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319811.976811:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319812.999809:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319812.999898:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319813.767393:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319813.767411:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319813.767415:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319813.767420:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319814.023323:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319814.023375:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319815.047331:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319815.047427:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319816.071393:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319816.071622:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319817.096138:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319817.097143:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319818.119422:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767319818.119676:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319818.888440:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319818.888458:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319818.888462:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319818.888466:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319819.143277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319819.143402:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319820.168145:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319820.168244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319821.193360:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319821.193461:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319822.215620:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319822.216841:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319823.239328:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319823.252927:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767319824.008191:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319824.008202:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319824.008205:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319824.008208:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319824.263247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319824.263323:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319825.287287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319825.291930:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319826.312208:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319826.312440:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319827.335235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319827.337341:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319828.103354:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319803/real 1767319803] req@ffff88c5c65cdf80 x1853163991651584/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319828 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319828.103438:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cdf80 x1853163991651584/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319828 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319828.103450:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319803/real 1767319803] req@ffff88c5c65cc000 x1853163991651712/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319828 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319828.103462:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cc000 x1853163991651712/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319828 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319828.103485:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319828.103527:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319828.103532:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991651584:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319828.103539:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319828.103545:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319828.103548:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991651712:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319828.103718:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319828.103725:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319828.103728:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319828.103734:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5002 00000100:00080000:3.0:1767319828.103748:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319828.103756:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319828.103811:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728380 x1853163991651840/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319828.103842:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319828.103845:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319828.103847:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319828.103851:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5002 00000100:00080000:3.0:1767319828.103856:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319828.103860:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319828.104215:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319828.104232:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728700 x1853163991651968/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319828.104320:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991651840:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319828.104445:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991651968:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767319828.360323:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319828.360633:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319829.383623:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319829.389766:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319830.409804:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319830.409821:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319831.432727:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319831.433433:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319832.455335:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319832.461792:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319833.224370:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319833.224386:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319833.224390:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319833.224394:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319833.479322:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319833.480382:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319834.503612:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319834.504427:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319835.527324:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319835.528861:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319836.551274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319836.551383:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319837.575548:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319837.575832:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319838.343627:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319838.343647:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319838.343650:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319838.343654:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319838.599535:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319838.602468:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319839.624116:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319839.624873:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319840.647558:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319840.647650:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319841.671494:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319841.671682:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319842.696599:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319842.696687:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319843.463240:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319843.463251:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319843.463273:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319843.463277:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319843.719280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319843.719488:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319844.743257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319844.746640:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319845.767350:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319845.767560:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319846.791502:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319846.792385:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319847.815427:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319847.815870:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319848.583376:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319848.583386:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319848.583389:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319848.583393:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319848.839875:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319848.840225:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319849.863221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319849.865317:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319850.887259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319850.887428:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319851.911234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319851.911499:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319852.935317:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319852.935815:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319853.703238:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319828/real 1767319828] req@ffff88c5c5728380 x1853163991651840/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319853 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319853.703277:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728380 x1853163991651840/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319853 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319853.703311:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319828/real 1767319828] req@ffff88c5c5728700 x1853163991651968/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319853 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319853.703325:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728700 x1853163991651968/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319853 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319853.703340:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319853.703409:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319853.703414:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991651840:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319853.703421:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319853.704563:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319853.704571:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767319853.704587:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319853.704594:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5027 00000100:00080000:1.0:1767319853.704606:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319853.704612:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767319853.704647:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf800 x1853163991652096/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767319853.704662:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319853.704665:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767319853.704667:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319853.704670:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5027 00000100:00080000:1.0:1767319853.704674:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319853.704677:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767319853.704687:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767319853.704764:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ce680 x1853163991652224/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319853.705160:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319853.705167:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991651968:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767319853.705201:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991652096:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319853.705343:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991652224:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767319853.959351:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319853.959619:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319854.983270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319854.983628:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319856.007499:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319856.007583:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319857.031310:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319857.031401:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319858.055261:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319858.055442:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319858.823174:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319858.823185:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319858.823188:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319858.823192:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319859.079280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319859.079297:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319860.103223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319860.103645:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319861.127508:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319861.127708:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319862.151205:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319862.152779:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319863.175564:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319863.177229:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767319863.943288:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319863.943299:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319863.943303:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319863.943307:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319864.199403:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319864.200073:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319865.223290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319865.229638:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319866.247434:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319866.247516:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319867.271727:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319867.271966:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319868.295289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319868.295790:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319869.063471:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319869.063486:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319869.063489:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319869.063493:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319869.319290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319869.319640:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319870.343434:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319870.344116:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319871.367286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319871.367519:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319872.391486:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319872.391926:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319873.415217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767319873.416916:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319874.183593:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319874.183603:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319874.183606:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319874.183626:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319874.440930:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319874.441633:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319875.463329:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319875.463375:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319876.487254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319876.487309:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319877.511239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319877.511338:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319878.280673:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319853/real 1767319853] req@ffff88c5c65cf800 x1853163991652096/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319878 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319878.280708:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf800 x1853163991652096/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319878 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319878.280718:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319853/real 1767319853] req@ffff88c5c65ce680 x1853163991652224/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319878 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319878.280730:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ce680 x1853163991652224/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319878 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319878.280768:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319878.280844:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319878.280849:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991652096:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319878.280855:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767319878.283120:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319878.283133:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319878.283138:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319878.283148:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5052 00000100:00080000:3.0:1767319878.283160:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319878.283167:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319878.283209:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729180 x1853163991652352/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319878.283228:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319878.283231:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319878.283233:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319878.283237:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5052 00000100:00080000:3.0:1767319878.283241:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319878.283245:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319878.283254:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319878.283266:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572b100 x1853163991652480/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1767319878.294918:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319878.294931:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991652224:192.168.204.145@tcp:38:kworker.0 00000100:00100000:0.0:1767319878.294978:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991652352:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319878.295111:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991652480:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767319878.535249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319878.535555:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319879.559355:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319879.564000:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319880.584195:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319880.585879:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319881.607228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319881.607651:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319882.631416:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319882.631647:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319883.399653:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319883.399666:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319883.399669:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319883.399673:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319883.655618:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319883.655997:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319884.679375:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319884.680007:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319885.703420:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319885.704484:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319886.727676:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319886.728265:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319887.752466:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319887.753083:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319888.519915:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319888.519927:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319888.519930:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319888.519933:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319888.775358:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319888.775630:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319889.799568:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319889.811303:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319890.823305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319890.824679:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319891.847217:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319891.847376:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319892.871526:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319892.871854:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319893.639273:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319893.639285:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319893.639288:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319893.639291:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319893.895235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319893.895985:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319894.919651:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319894.919910:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319895.943242:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319895.943380:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319896.967294:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319896.968257:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319897.991611:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319897.992614:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319898.759574:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319898.759589:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319898.759593:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319898.759596:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319899.015999:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319899.034579:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319900.039390:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319900.039440:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319901.063250:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319901.063513:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319902.088486:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319902.088515:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319903.111249:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319903.111462:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319903.879248:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319903.879260:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319903.879262:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319903.879266:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319903.879828:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319878/real 1767319878] req@ffff88c5c5729180 x1853163991652352/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319903 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319903.879864:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729180 x1853163991652352/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319903 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319903.879874:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319878/real 1767319878] req@ffff88c5c572b100 x1853163991652480/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319903 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1767319903.879886:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572b100 x1853163991652480/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319903 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767319903.879902:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319903.879920:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319903.879924:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991652352:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767319903.879931:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767319903.879936:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:2.0:1767319903.879939:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991652480:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319903.879995:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319903.879999:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319903.880002:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319903.880006:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5077 00000100:00080000:3.0:1767319903.880016:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319903.880060:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319903.880087:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5728000 x1853163991652608/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319903.880127:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319903.880130:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319903.880132:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319903.880136:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5077 00000100:00080000:3.0:1767319903.880139:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319903.880143:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319903.880150:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319903.880160:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572ad80 x1853163991652736/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767319903.880222:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991652608:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767319903.880318:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991652736:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767319904.135230:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319904.138922:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319905.159323:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319905.159933:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319906.183442:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319906.183513:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319907.207630:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319907.207687:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319908.231849:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319908.232008:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767319909.000196:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319909.000208:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319909.000210:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319909.000214:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319909.255371:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319909.256387:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319910.280255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319910.284780:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319911.303399:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319911.304190:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319912.327282:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319912.328990:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319913.351245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319913.351602:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319914.119409:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319914.119419:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319914.119422:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319914.119426:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319914.375174:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319914.375388:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319915.399406:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319915.399511:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319916.423368:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319916.425182:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319917.448271:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319917.448401:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319918.471248:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319918.471581:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319919.239242:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319919.239253:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319919.239256:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319919.239260:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319919.496335:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319919.496432:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319920.519546:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319920.521154:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319921.543323:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319921.543598:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319922.567399:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319922.567474:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319923.591409:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319923.591487:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319924.359910:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319924.359920:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319924.359922:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319924.359926:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319924.615239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319924.616850:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319925.639350:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319925.639474:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319926.663472:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319926.663669:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319927.688912:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319927.689729:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767319928.455253:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319903/real 1767319903] req@ffff88c5c5728000 x1853163991652608/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319928 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319928.455321:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5728000 x1853163991652608/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319928 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319928.455331:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319903/real 1767319903] req@ffff88c5c572ad80 x1853163991652736/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319928 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1767319928.455342:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572ad80 x1853163991652736/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319928 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:1.0:1767319928.455373:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319928.455404:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319928.455408:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5728000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991652608:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767319928.455414:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1767319928.455419:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:1.0:1767319928.455422:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991652736:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319928.455528:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319928.455534:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319928.455543:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319928.455549:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5103 00000100:00080000:3.0:1767319928.455558:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319928.455564:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319928.455593:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729f80 x1853163991652864/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319928.455619:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319928.455622:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319928.455623:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319928.455628:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5103 00000100:00080000:3.0:1767319928.455631:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319928.455635:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319928.455644:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319928.455653:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729c00 x1853163991652992/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1767319928.455783:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991652864:192.168.204.145@tcp:250:kworker.0 00000100:00100000:1.0:1767319928.455883:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991652992:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767319928.711381:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319928.711592:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319929.735809:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319929.736032:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319930.759472:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319930.759767:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319931.783700:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319931.783852:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319932.807206:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319932.807297:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319933.575264:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319933.575276:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319933.575279:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319933.575282:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767319933.832157:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319933.834380:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319934.855367:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319934.856653:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319935.879210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319935.879413:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319936.903296:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319936.903690:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319937.927941:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319937.929155:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319938.695293:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319938.695303:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319938.695306:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319938.695310:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319938.951223:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319938.951326:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319939.975337:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319939.975687:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319940.999347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319940.999769:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319942.023311:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319942.023658:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319943.047226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319943.049707:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319943.815212:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319943.815222:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319943.815225:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319943.815228:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319944.071350:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319944.071760:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319945.095493:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319945.097327:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319946.120558:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319946.120581:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319947.143395:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319947.143569:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319948.167216:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319948.167419:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319948.935213:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319948.935224:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319948.935228:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319948.935250:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319949.191235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319949.191324:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319950.219085:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319950.219754:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319951.239247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319951.240125:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319952.263215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319952.263315:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319953.031242:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319928/real 1767319928] req@ffff88c5c5729f80 x1853163991652864/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319953 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319953.031289:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729f80 x1853163991652864/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319953 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319953.031332:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319928/real 1767319928] req@ffff88c5c5729c00 x1853163991652992/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319953 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319953.031346:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729c00 x1853163991652992/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319953 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319953.031444:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319953.031574:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319953.031579:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991652864:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319953.031586:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319953.031593:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319953.031596:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991652992:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767319953.031648:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319953.031654:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1767319953.031657:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319953.031662:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5127 00000100:00080000:3.0:1767319953.031671:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319953.031676:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:3.0:1767319953.031706:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c572aa00 x1853163991653120/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767319953.031734:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319953.031737:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1767319953.031738:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1767319953.031742:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5127 00000100:00080000:3.0:1767319953.031746:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1767319953.031750:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:3.0:1767319953.031760:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1767319953.031772:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c5729500 x1853163991653248/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319953.031840:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991653120:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319953.032141:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991653248:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767319953.287166:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319953.287455:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319954.311298:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319954.311871:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319955.336146:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319955.336170:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319956.359431:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767319956.368493:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767319957.383302:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319957.383731:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767319958.152054:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319958.152069:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319958.152073:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319958.152077:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319958.408625:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319958.408945:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319959.431390:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319959.431482:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319960.455239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319960.458559:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319961.479269:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319961.483747:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319962.503359:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319962.505155:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767319963.271215:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319963.271227:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767319963.271231:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767319963.271234:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319963.527340:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319963.527715:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319964.551275:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319964.560476:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319965.575930:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319965.575948:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319966.599364:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319966.608313:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319967.623282:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319967.626806:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767319968.391220:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319968.391232:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319968.391235:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319968.391238:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767319968.647449:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319968.648368:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319969.672221:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319969.672287:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319970.695320:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319970.697562:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319971.719652:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319971.719731:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319972.743841:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319972.743978:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767319973.511580:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319973.511591:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319973.511594:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319973.511597:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319973.767504:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319973.768070:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319974.791347:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319974.792141:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319975.815259:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319975.815973:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767319976.839307:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319976.839450:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319977.863350:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319977.863550:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319978.631273:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319953/real 1767319953] req@ffff88c5c572aa00 x1853163991653120/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319978 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319978.631345:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c572aa00 x1853163991653120/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767319978 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319978.631359:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319953/real 1767319953] req@ffff88c5c5729500 x1853163991653248/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319978 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767319978.631373:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c5729500 x1853163991653248/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767319978 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767319978.631395:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319978.631407:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319978.631412:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c572aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991653120:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767319978.631418:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767319978.631422:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767319978.631425:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c5729500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991653248:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767319978.635503:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319978.635515:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767319978.635518:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319978.635529:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5152 00000100:00080000:1.0:1767319978.635540:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319978.635546:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767319978.635586:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd500 x1853163991653376/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767319978.635617:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319978.635621:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767319978.635623:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767319978.635627:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5152 00000100:00080000:1.0:1767319978.635631:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767319978.635635:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767319978.635645:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767319978.635657:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd180 x1853163991653504/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767319978.638671:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991653376:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767319978.638872:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991653504:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767319978.887493:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319978.890051:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319979.911337:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319979.912240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767319980.935866:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319980.936007:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319981.959765:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319981.959915:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319982.984602:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319982.984780:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319983.751370:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319983.751392:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319983.751396:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319983.751399:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767319984.007513:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319984.007817:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319985.031274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319985.031489:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319986.055511:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319986.055786:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319987.079418:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319987.080933:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319988.103247:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319988.104406:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767319988.871829:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319988.871841:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767319988.871844:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767319988.871848:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319989.127222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319989.127480:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319990.152165:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319990.152244:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767319991.175221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319991.177551:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767319992.199820:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319992.207784:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319993.223309:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319993.226111:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767319993.991325:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319993.991336:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767319993.991339:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767319993.991342:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319994.247374:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319994.248298:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319995.271204:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319995.271595:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319996.295574:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319996.300319:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767319997.321190:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767319997.321840:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767319998.343327:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319998.345256:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767319999.111545:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319999.111556:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767319999.111559:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767319999.111562:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767319999.367273:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767319999.367504:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320000.391206:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320000.391644:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320001.415210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320001.415317:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320002.439672:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320002.441669:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767320003.209214:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319978/real 1767319978] req@ffff88c5c65cd500 x1853163991653376/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320003 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320003.209250:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd500 x1853163991653376/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320003 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320003.209261:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767319978/real 1767319978] req@ffff88c5c65cd180 x1853163991653504/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320003 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320003.209272:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd180 x1853163991653504/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320003 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767320003.209290:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320003.209319:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320003.209324:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991653376:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767320003.209330:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320003.209336:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320003.209367:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991653504:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767320003.209556:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320003.209561:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767320003.209564:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767320003.209569:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5177 00000100:00080000:1.0:1767320003.209579:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767320003.209585:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767320003.209616:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cdc00 x1853163991653632/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767320003.209653:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320003.209656:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767320003.209658:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767320003.209662:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5177 00000100:00080000:1.0:1767320003.209665:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767320003.209669:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767320003.209678:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767320003.209691:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd180 x1853163991653760/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767320003.210140:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991653632:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767320003.210255:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991653760:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767320003.463197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320003.463280:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320004.487288:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320004.487491:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320005.511308:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320005.512115:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320006.535369:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320006.535449:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320007.559214:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320007.565723:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767320008.327326:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320008.327338:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320008.327341:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320008.327344:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320008.586404:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320008.587881:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320009.608643:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320009.610375:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320010.631870:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320010.632413:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320011.655438:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320011.657144:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320012.679219:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320012.679787:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767320013.448339:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320013.448351:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767320013.448354:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320013.448357:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320013.703277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320013.707591:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320014.727315:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320014.727776:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320015.751785:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320015.751869:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320016.775210:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320016.775345:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320017.800167:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320017.800704:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767320018.569949:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320018.569962:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320018.569966:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320018.569969:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767320018.823364:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320018.823428:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320019.847455:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320019.849174:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320020.871323:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320020.871486:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320021.895433:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320021.895602:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320022.919489:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320022.920136:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767320023.689111:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320023.689123:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320023.689126:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320023.689129:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767320023.943404:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320023.944280:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320024.967290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320024.967402:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320025.991218:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320025.991402:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320027.015221:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320027.015700:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320028.039194:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320028.045309:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767320028.807291:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320028.807302:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320028.807304:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320028.807308:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767320028.807399:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320003/real 1767320003] req@ffff88c5c65cdc00 x1853163991653632/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320028 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320028.807430:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cdc00 x1853163991653632/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320028 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320028.807440:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320003/real 1767320003] req@ffff88c5c65cd180 x1853163991653760/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320028 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320028.807451:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd180 x1853163991653760/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320028 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767320028.807469:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320028.807495:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320028.807499:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991653632:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767320028.807505:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320028.807510:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320028.807513:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991653760:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767320028.807661:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320028.807666:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767320028.807669:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320028.807675:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5202 00000100:00080000:2.0:1767320028.807683:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320028.807690:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767320028.807740:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4e00 x1853163991653888/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767320028.807767:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320028.807771:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767320028.807772:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320028.807776:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5202 00000100:00080000:2.0:1767320028.807780:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320028.807784:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767320028.807793:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767320028.807815:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5500 x1853163991654016/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767320028.807884:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991653888:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767320028.807994:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991654016:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767320029.063424:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320029.063844:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320030.087255:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320030.089420:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320031.111462:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320031.111897:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320032.136156:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320032.136212:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320033.159245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320033.159431:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767320033.927726:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320033.927736:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767320033.927739:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320033.927742:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767320034.183485:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320034.183836:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320035.207303:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320035.207475:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320036.234184:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767320036.234262:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767320037.255293:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320037.255309:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767320038.279279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320038.279404:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767320039.050372:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320039.050386:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767320039.050389:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320039.050394:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320039.303378:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320039.303556:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320040.327263:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320040.329463:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320041.351420:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320041.352559:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320042.375251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320042.375321:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320043.399488:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320043.401004:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767320044.167604:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320044.167619:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767320044.167622:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320044.167626:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320044.423305:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320044.423627:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320045.447381:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320045.448476:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767320046.475419:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767320046.476060:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:0.0:1767320047.495295:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320047.495629:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767320048.519267:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320048.519340:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767320049.288070:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320049.288313:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320049.288316:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320049.288319:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:1.0:1767320049.543289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320049.543888:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1767320050.567342:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320050.567680:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320051.591547:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320051.592418:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320052.615611:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320052.616313:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767320053.383725:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320028/real 1767320028] req@ffff88c5c57d4e00 x1853163991653888/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320053 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320053.383776:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4e00 x1853163991653888/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320053 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320053.383789:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320028/real 1767320028] req@ffff88c5c57d5500 x1853163991654016/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320053 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320053.383801:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5500 x1853163991654016/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320053 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767320053.383827:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320053.383841:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320053.383846:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991653888:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767320053.383854:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320053.383888:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320053.383892:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991654016:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767320053.389598:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320053.389614:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767320053.389618:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767320053.389632:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5227 00000100:00080000:1.0:1767320053.389647:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767320053.389657:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767320053.389710:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cd500 x1853163991654144/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767320053.389751:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320053.389756:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767320053.389758:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767320053.389762:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5227 00000100:00080000:1.0:1767320053.389766:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767320053.389770:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767320053.389781:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767320053.389794:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65ce680 x1853163991654272/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1767320053.390185:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991654144:192.168.204.145@tcp:250:kworker.0 00000100:00100000:2.0:1767320053.390332:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991654272:192.168.204.145@tcp:38:kworker.0 00010000:00010000:3.0:1767320053.639621:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320053.640069:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320054.663331:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320054.663591:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320055.690842:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320055.691240:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320056.711597:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320056.711910:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320057.735266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320057.736920:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767320058.503268:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320058.503305:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320058.503308:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320058.503312:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767320058.761844:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320058.762361:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320059.783224:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320059.783471:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320060.807381:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320060.807535:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320061.831611:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320061.832354:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767320062.855549:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320062.855734:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767320063.623237:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320063.623249:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320063.623253:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320063.623256:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320063.879289:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320063.879723:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320064.903282:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320064.903449:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320065.927266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320065.927357:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320066.951225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320066.951702:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320067.985114:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320067.986279:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1767320068.745743:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320068.745751:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767320068.745754:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320068.745756:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767320069.000660:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320069.000986:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320070.023366:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320070.023595:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320071.047516:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320071.047871:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320072.071254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320072.071617:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320073.096417:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320073.096847:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767320073.865984:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320073.865994:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320073.865996:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320073.866000:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320074.119278:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320074.119280:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767320075.143384:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320075.143460:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320076.167254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320076.167663:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320077.191301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320077.191498:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320078.215243:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320078.216487:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767320078.983824:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320078.983833:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320078.983836:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320078.983838:0:19117:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:0.0:1767320078.984796:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320053/real 1767320053] req@ffff88c5c65cd500 x1853163991654144/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320078 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320078.984826:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cd500 x1853163991654144/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320078 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320078.984833:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320053/real 1767320053] req@ffff88c5c65ce680 x1853163991654272/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320078 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320078.984841:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65ce680 x1853163991654272/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320078 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767320078.984855:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320078.984873:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320078.984876:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991654144:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767320078.984880:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320078.984885:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320078.984889:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65ce680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991654272:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767320078.984969:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320078.984985:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767320078.984989:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320078.984994:0:19117:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5252 00000100:00080000:2.0:1767320078.985002:0:19117:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320078.985008:0:19117:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767320078.985037:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6300 x1853163991654400/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767320078.985057:0:19117:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320078.985059:0:19117:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767320078.985061:0:19117:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320078.985063:0:19117:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5252 00000100:00080000:2.0:1767320078.985066:0:19117:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320078.985069:0:19117:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767320078.985077:0:19117:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767320078.985085:0:19117:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7800 x1853163991654528/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767320078.985234:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991654400:192.168.204.145@tcp:250:kworker.0 00000100:00100000:0.0:1767320078.985335:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991654528:192.168.204.145@tcp:38:kworker.0 00010000:00010000:2.0:1767320079.241117:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320079.242153:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320080.265460:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320080.265519:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320081.287246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320081.288245:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320082.311229:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320082.311608:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767320083.335292:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320083.335384:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767320084.103384:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320084.103396:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767320084.103399:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320084.103402:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767320084.359390:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320084.359466:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320085.383234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320085.384174:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320086.408914:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320086.409787:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320087.432404:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320087.433122:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767320088.455282:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320088.455377:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767320089.223319:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320089.223331:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767320089.223334:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320089.223358:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320089.482069:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320089.482337:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320090.503853:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320090.504814:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320091.527226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320091.527435:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320092.551213:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320092.552779:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320093.575364:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320093.576890:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:1.0:1767320094.344446:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320094.344456:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320094.344459:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320094.344462:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320094.599382:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320094.602039:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320095.623197:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320095.623374:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320096.647338:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320096.647502:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320097.673310:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320097.673489:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320098.695222:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320098.698092:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767320099.463238:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320099.463246:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320099.463248:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320099.463250:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:2.0:1767320099.719370:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320099.719476:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320100.743998:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320100.744649:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320101.767284:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320101.769767:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320102.791281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320102.791608:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1767320103.563308:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320078/real 1767320078] req@ffff88c5c57d6300 x1853163991654400/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320103 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320103.563336:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6300 x1853163991654400/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320103 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320103.563359:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320078/real 1767320078] req@ffff88c5c57d7800 x1853163991654528/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320103 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1767320103.563368:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7800 x1853163991654528/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320103 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1767320103.563381:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320103.563400:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320103.563403:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991654400:192.168.204.145@tcp:250:kworker.0 00000100:00080000:0.0:1767320103.563407:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1767320103.563411:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:0.0:1767320103.563413:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991654528:192.168.204.145@tcp:38:kworker.0 00000100:00080000:1.0:1767320103.563811:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320103.563818:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767320103.563822:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767320103.563905:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5278 00000100:00080000:1.0:1767320103.563916:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767320103.563923:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767320103.563966:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cf800 x1853163991654656/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767320103.564029:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991654656:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767320103.564460:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320103.564465:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767320103.564467:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767320103.564472:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5278 00000100:00080000:1.0:1767320103.564476:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767320103.564480:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767320103.564499:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767320103.564513:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cc000 x1853163991654784/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1767320103.564988:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991654784:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767320103.815352:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320103.816906:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767320104.839296:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320104.840061:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320105.863322:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320105.866484:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320106.888083:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320106.888285:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320107.911281:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320107.911537:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767320108.679202:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320108.679211:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320108.679214:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320108.679216:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320108.935270:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320108.935427:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320109.959402:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320109.960007:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320110.983290:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320110.983680:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320112.007257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320112.007438:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1767320113.031306:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320113.031396:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:3.0:1767320113.799278:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320113.799291:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:3.0:1767320113.799327:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1767320113.799331:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320114.055260:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320114.055421:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320115.079286:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320115.079406:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320116.103232:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320116.103499:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320117.127225:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320117.127505:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320118.151411:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320118.152020:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767320118.919568:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320118.919579:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320118.919582:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320118.919603:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320119.175815:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320119.176113:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320120.199829:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320120.201252:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320121.223421:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320121.223843:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320122.247226:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320122.248267:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320123.271424:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320123.271480:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767320124.039322:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320124.039333:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320124.039336:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320124.039339:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320124.295279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767320124.295313:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320125.319228:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320125.320399:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320126.343211:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320126.343478:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320127.367630:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320127.368876:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320128.135237:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320103/real 1767320103] req@ffff88c5c65cc000 x1853163991654784/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320128 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320128.135273:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cc000 x1853163991654784/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320128 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320128.135284:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320103/real 1767320103] req@ffff88c5c65cf800 x1853163991654656/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320128 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320128.135297:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cf800 x1853163991654656/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320128 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767320128.135348:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767320128.135370:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767320128.135375:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991654784:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767320128.135401:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767320128.135408:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767320128.135412:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991654656:192.168.204.145@tcp:250:kworker.0 00000100:00080000:1.0:1767320128.135635:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320128.135642:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1767320128.135652:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767320128.135658:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5302 00000100:00080000:1.0:1767320128.135667:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767320128.135674:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:1.0:1767320128.135709:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cdf80 x1853163991654912/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1767320128.135735:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320128.135739:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1767320128.135740:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1767320128.135744:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5302 00000100:00080000:1.0:1767320128.135747:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1767320128.135750:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:1.0:1767320128.135759:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1767320128.135771:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c65cca80 x1853163991655040/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767320128.135783:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991654912:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767320128.135883:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991655040:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767320128.391601:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767320128.393668:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320129.415297:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320129.415419:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320130.439277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320130.445048:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320131.463337:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320131.464885:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767320132.488269:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320132.488868:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767320133.255277:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320133.255290:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320133.255293:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320133.255296:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320133.511308:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320133.516878:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320134.535263:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320134.535524:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320135.559207:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320135.559387:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320136.583175:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767320136.584591:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320137.607251:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320137.607477:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1767320138.376296:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320138.376308:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:1.0:1767320138.376310:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1767320138.376314:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320138.631441:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320138.633632:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320139.655420:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320139.655813:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:0.0:1767320140.679233:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320140.679472:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320141.703572:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320141.703726:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320142.727352:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320142.727536:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767320143.495617:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320143.495629:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320143.495632:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320143.495658:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320143.751620:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320143.753816:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320144.775203:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320144.775245:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320145.799194:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:2.0:1767320145.799446:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320146.823301:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320146.824432:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320147.847354:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320147.847532:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767320148.619333:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320148.619346:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320148.619349:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320148.619353:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320148.871744:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320148.871882:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320149.895246:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320149.897024:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320150.922919:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320150.927595:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320151.943199:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320151.943483:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320152.968258:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320152.968467:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767320153.735233:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320153.735249:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320153.735249:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320128/real 1767320128] req@ffff88c5c65cca80 x1853163991655040/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320153 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1767320153.735255:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320153.735263:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320153.735297:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cca80 x1853163991655040/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320153 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320153.735309:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320128/real 1767320128] req@ffff88c5c65cdf80 x1853163991654912/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320153 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320153.735320:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c65cdf80 x1853163991654912/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320153 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767320153.735349:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767320153.735370:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767320153.735375:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991655040:192.168.204.145@tcp:38:kworker.0 00000100:00080000:3.0:1767320153.735381:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767320153.735385:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767320153.735388:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c65cdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991654912:192.168.204.145@tcp:250:kworker.0 00000100:00080000:2.0:1767320153.735522:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320153.735527:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767320153.735530:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320153.735536:0:8649:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5327 00000100:00080000:2.0:1767320153.735547:0:8649:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320153.735581:0:8649:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767320153.735625:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4000 x1853163991655168/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767320153.735658:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320153.735681:0:8649:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767320153.735683:0:8649:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320153.735687:0:8649:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5327 00000100:00080000:2.0:1767320153.735692:0:8649:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320153.735696:0:8649:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767320153.735714:0:8649:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767320153.735726:0:8649:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d6a00 x1853163991655296/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767320153.735966:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991655168:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767320153.736071:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991655296:192.168.204.145@tcp:38:kworker.0 00000100:00100000:2.0:1767320153.991303:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320153.993887:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320155.015326:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320155.017643:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320156.039236:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320156.040148:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320157.064507:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320157.064888:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320158.087324:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320158.090652:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767320158.855476:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320158.855488:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767320158.855491:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320158.855494:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320159.111336:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320159.111562:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320160.135215:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:3.0:1767320160.135642:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1767320161.160684:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320161.160759:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320162.183645:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320162.186784:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320163.209950:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320163.210021:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:2.0:1767320163.975269:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320163.975280:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320163.975283:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320163.975287:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:3.0:1767320164.231355:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320164.231432:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320165.255520:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320165.257403:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320166.279463:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320166.279710:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320167.303427:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320167.303475:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1767320168.327446:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320168.327477:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767320169.096545:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320169.096555:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767320169.096558:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320169.096560:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00010000:00010000:1.0:1767320169.351474:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320169.351604:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320170.375274:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320170.382261:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320171.399254:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320171.399366:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320172.424134:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320172.427866:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320173.447245:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320173.448603:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767320174.215294:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320174.215306:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320174.215309:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320174.215312:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320174.471231:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320174.472005:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320175.495321:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320175.495342:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320176.519269:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320176.523433:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320177.543360:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320177.543891:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320178.311698:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320153/real 1767320153] req@ffff88c5c57d4000 x1853163991655168/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320178 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320178.312482:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d4000 x1853163991655168/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320178 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320178.312495:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320153/real 1767320153] req@ffff88c5c57d6a00 x1853163991655296/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320178 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320178.312508:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d6a00 x1853163991655296/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320178 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767320178.312527:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767320178.312681:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767320178.312686:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991655168:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767320178.312692:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1767320178.320023:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320178.320034:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767320178.320039:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320178.320048:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5352 00000100:00080000:2.0:1767320178.320059:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320178.320066:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767320178.320132:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d7480 x1853163991655424/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767320178.320151:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320178.320154:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767320178.320156:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320178.320159:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5352 00000100:00080000:2.0:1767320178.320163:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320178.320167:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767320178.320185:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767320178.320197:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5f80 x1853163991655552/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1767320178.320241:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767320178.320247:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991655296:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767320178.320287:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991655424:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767320178.320410:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991655552:192.168.204.145@tcp:38:kworker.0 00010000:00010000:1.0:1767320178.567514:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320178.568474:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320179.591687:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320179.594235:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320180.615489:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320180.616249:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320181.639546:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320181.639628:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320182.663239:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320182.666402:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1767320183.431419:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320183.431430:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767320183.431433:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320183.431437:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320183.687258:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320183.689285:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320184.711227:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320184.714256:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320185.735468:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320185.735567:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320186.759341:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320186.759700:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320187.783417:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320187.783963:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00080000:0.0:1767320188.551366:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320188.551376:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:0.0:1767320188.551379:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1767320188.551383:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:3.0:1767320188.807234:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320188.808889:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320189.831328:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320189.831924:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320190.855349:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320190.858257:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320191.879272:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320191.879365:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320192.903266:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320192.905242:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767320193.671533:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320193.671543:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320193.671545:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320193.671549:0:8:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320193.927469:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320193.927511:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320194.952779:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320194.954972:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320195.975482:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320195.975618:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:2.0:1767320196.999657:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320197.001892:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320198.023347:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320198.023387:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1767320198.791166:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320198.791181:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00080000:2.0:1767320198.791184:0:8649:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level CONNECTING/4 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320198.791188:0:8649:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: not pinging (in recovery or recovery disabled: CONNECTING) 00000100:00100000:2.0:1767320199.047277:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320199.048438:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320200.071279:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320200.075206:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320201.095299:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320201.096452:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1767320202.119291:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320202.124483:0:7227:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1767320203.145398:0:7228:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320203.145832:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00000100:00100000:3.0:1767320203.911287:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320178/real 1767320178] req@ffff88c5c57d7480 x1853163991655424/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320203 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320203.911337:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d7480 x1853163991655424/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 1 dl 1767320203 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320203.911352:0:2485:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1767320178/real 1767320178] req@ffff88c5c57d5f80 x1853163991655552/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320203 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1767320203.911365:0:2485:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff88c5c57d5f80 x1853163991655552/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 1 dl 1767320203 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1767320203.911390:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767320203.911422:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767320203.911428:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991655424:192.168.204.145@tcp:250:kworker.0 00000100:00080000:3.0:1767320203.911436:0:2485:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1767320203.911445:0:2485:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.145@tcp failed (-110) 00000100:00100000:3.0:1767320203.911475:0:2485:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff88c5c57d5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991655552:192.168.204.145@tcp:38:kworker.0 00000100:00080000:2.0:1767320203.911856:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) cf740287-d7f4-4a51-9260-7c6190205bc2->MGS: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320203.911864:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1767320203.911868:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c9ce5800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320203.911874:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.145@tcp: connect to NID 192.168.204.145@tcp last attempt 5377 00000100:00080000:2.0:1767320203.911888:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.145@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320203.911904:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.145@tcp: import ffff88c5c9ce5800 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000100:00100000:2.0:1767320203.911948:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d5c00 x1853163991655680/t0(0) o250->MGC192.168.204.145@tcp@192.168.204.145@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1767320203.911975:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5398f0ac-a698-45f7-862d-28236c50e3c9->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1767320203.911979:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1767320203.911981:0:8:0:(import.c:65:import_set_state_nolock()) ffff88c5c6dc1000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1767320203.911985:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: connect to NID 192.168.204.145@tcp last attempt 5377 00000100:00080000:2.0:1767320203.911990:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1767320203.911994:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff88c5c68bd000: import ffff88c5c6dc1000 using connection 192.168.204.145@tcp/192.168.204.145@tcp 00000008:00100000:2.0:1767320203.912006:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34668544 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1767320203.912018:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff88c5c57d4a80 x1853163991655808/t0(0) o38->lustre-MDT0000-mdc-ffff88c5c68bd000@192.168.204.145@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1767320203.912126:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:cf740287-d7f4-4a51-9260-7c6190205bc2:2485:1853163991655680:192.168.204.145@tcp:250:kworker.0 00000100:00100000:3.0:1767320203.912256:0:2485:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff88c5c57d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5398f0ac-a698-45f7-862d-28236c50e3c9:2485:1853163991655808:192.168.204.145@tcp:38:kworker.0 00000100:00100000:3.0:1767320204.167378:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320204.169332:0:15085:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1767320205.192066:0:48036:0:(client.c:2690:ptlrpc_set_wait()) set ffff88c5d0fe8480 going to sleep for 0 seconds 00010000:00010000:1.0:1767320205.192490:0:7704:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 20531 lines, 20531 kept, 0 dropped, 0 bad.