00010000:00010000:2.0F:1782488020.461964:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0F:1782488021.487029:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0F:1782488022.510230:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0F:1782488023.535085:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488024.494019:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52de680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298627584:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488024.494191:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298627840:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488024.494237:0:2362:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1782488024.494244:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:60894528-77cf-40cf-92ce-b54cc4bbae30:2362:1869066298627456:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488024.494272:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298627712:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488024.506385:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:60894528-77cf-40cf-92ce-b54cc4bbae30:2362:1869066298627456:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488024.506413:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298627712:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488024.506427:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52de680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298627584:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488024.506439:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298627840:192.168.202.121@tcp:400:kworker.0 00010000:00010000:0.0:1782488024.558995:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488025.582130:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488026.606827:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488027.632613:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488028.654127:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488029.616506:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298627968:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488029.616651:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628224:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488029.616699:0:2363:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1782488029.616707:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628096:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488029.616762:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628352:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488029.621932:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628096:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488029.621954:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628352:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488029.621965:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298627968:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488029.621976:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628224:192.168.202.121@tcp:400:kworker.0 00010000:00010000:0.0:1782488029.681475:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488030.702338:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488031.725954:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488032.751084:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488033.774952:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488034.733963:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298628480:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488034.734115:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628736:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488034.734150:0:2363:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1782488034.734157:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628608:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488034.734255:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628864:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488034.739039:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298628480:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488034.740498:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628608:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488034.740522:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628864:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488034.740534:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298628736:192.168.202.121@tcp:400:kworker.0 00010000:00010000:3.0:1782488034.797898:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488035.830090:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1782488035.876226:0:320079:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 110: do not grant another lock on resend ========================================================== 11:33:54 (1782488034) 00010000:00010000:3.0:1782488036.846079:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488037.869990:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488038.893950:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488039.854111:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298628992:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488039.854208:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e95c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629248:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488039.854236:0:2363:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1782488039.854242:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629120:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488039.854261:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629376:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488039.863753:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e95c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629248:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488039.863788:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629120:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488039.863799:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629376:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488039.863810:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298628992:192.168.202.121@tcp:400:kworker.0 00010000:00010000:3.0:1782488039.921012:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488040.942014:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488041.965959:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488042.994287:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488044.016930:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488044.975407:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298629504:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488044.975549:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629760:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488044.975584:0:2363:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1782488044.975591:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629632:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488044.975617:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e95c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629888:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488044.988497:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629632:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488044.988516:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e95c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629888:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488044.988538:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298629504:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488044.988551:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298629760:192.168.202.121@tcp:400:kworker.0 00010000:00010000:3.0:1782488045.043839:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488046.067092:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488047.086377:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488048.110006:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488049.138763:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488050.095502:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298630016:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488050.095608:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e95c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298630272:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488050.095647:0:2363:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1782488050.095829:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298630144:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488050.095859:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298630400:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488050.103437:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298630144:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488050.103458:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298630400:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488050.103470:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e95c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298630272:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488050.109790:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e96a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298630016:192.168.202.121@tcp:400:kworker.0 00010000:00010000:3.0:1782488050.157993:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488051.182179:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488052.207167:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488053.231189:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488053.677861:0:294639:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6aebfb3000 going to sleep for 1 seconds 00010000:00010000:3.0:1782488054.255117:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488054.701846:0:294639:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6aebfb3000 going to sleep for 1 seconds 00000100:00100000:3.0:1782488055.215819:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298630656:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.215933:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298630912:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.215972:0:2362:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1782488055.215979:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:60894528-77cf-40cf-92ce-b54cc4bbae30:2362:1869066298630528:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.216005:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298630784:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.227063:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298630784:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.227085:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298630656:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.227097:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298630912:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.227321:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:60894528-77cf-40cf-92ce-b54cc4bbae30:2362:1869066298630528:192.168.202.121@tcp:400:kworker.0 00010000:00010000:3.0:1782488055.278541:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000400:2.0:1782488055.725862:0:294639:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1782487586/real 1782487586] req@ffff8a6adf730e00 x1869066294400128/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/1152 e 3 to 1 dl 1782488055 ref 2 fl Rpc:XQr/600/ffffffff rc 0/-1 job:'rm.500' uid:500 gid:500 projid:0 00000100:00080000:2.0:1782488055.768459:0:294639:0:(import.c:65:import_set_state_nolock()) ffff8a6ad1018000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1782488055.768474:0:294639:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-mdc-ffff8a6ac800b000: Connection to lustre-MDT0000 (at 192.168.202.121@tcp) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1782488055.780443:0:294639:0:(import.c:418:ptlrpc_pinger_force()) lustre-MDT0000_UUID: waking up pinger s:DISCONN 00000100:00100000:2.0:1782488055.780480:0:294639:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6aebfb3000 going to sleep for 0 seconds 00000100:00080000:3.0:1782488055.780886:0:319644:0:(pinger.c:231:ptlrpc_pinger_process_import()) 5c4ca15f-f34e-4111-b105-de67ffe3ec92->lustre-MDT0000_UUID: level DISCONN/3 force 1 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488055.780892:0:319644:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488055.780895:0:319644:0:(import.c:65:import_set_state_nolock()) ffff8a6ad1018000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488055.780904:0:319644:0:(import.c:512:import_select_connection()) lustre-MDT0000-mdc-ffff8a6ac800b000: connect to NID 192.168.202.121@tcp last attempt 6675 00000100:00080000:3.0:1782488055.780931:0:319644:0:(import.c:626:import_select_connection()) lustre-MDT0000-mdc-ffff8a6ac800b000: import ffff8a6ad1018000 using connection 192.168.202.121@tcp/192.168.202.121@tcp 00000008:00100000:3.0:1782488055.780941:0:319644:0:(osc_request.c:3829:osc_reconnect()) ocd_connect_flags: 0xae7e7e7be374d3b8 ocd_version: 34682463 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1782488055.780959:0:319644:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff8a6af52df800 x1869066298631168/t0(0) o38->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488055.781017:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298631040:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.781105:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298631424:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.781150:0:2363:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1782488055.781157:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298631296:192.168.202.121@tcp:400:kworker.0 00000100:00100000:1.0:1782488055.781508:0:2359:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2359:1869066298631168:192.168.202.121@tcp:38:kworker.0 00000100:00100000:3.0:1782488055.784503:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298631296:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.784520:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298631040:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488055.784532:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298631424:192.168.202.121@tcp:400:kworker.0 00000100:00080000:1.0:1782488055.791670:0:2359:0:(import.c:1134:ptlrpc_connect_interpret()) lustre-MDT0000-mdc-ffff8a6ac800b000: connect to target with instance 2 00000100:00080000:1.0:1782488055.791711:0:2359:0:(import.c:1299:ptlrpc_connect_interpret()) reconnected to lustre-MDT0000_UUID@192.168.202.121@tcp after partition 00000100:00080000:1.0:1782488055.791715:0:2359:0:(import.c:65:import_set_state_nolock()) ffff8a6ad1018000 lustre-MDT0000_UUID: changing import state from CONNECTING to RECOVER 00000100:00080000:1.0:1782488055.791744:0:2359:0:(import.c:65:import_set_state_nolock()) ffff8a6ad1018000 lustre-MDT0000_UUID: changing import state from RECOVER to FULL 00000100:02000000:1.0:1782488055.793813:0:2359:0:(import.c:1744:ptlrpc_import_recovery_state_machine()) lustre-MDT0000-mdc-ffff8a6ac800b000: Connection restored to 192.168.202.121@tcp (at 192.168.202.121@tcp) 00000100:00080000:1.0:1782488055.799370:0:2359:0:(recover.c:191:ptlrpc_wake_delayed()) @@@ waking (set 000000002ba5b0b8): req@ffff8a6adf730e00 x1869066294400128/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/1152 e 3 to 1 dl 1782488055 ref 2 fl Rpc:XQU/600/ffffffff rc 0/-1 job:'rm.500' uid:500 gid:500 projid:0 00000100:00080000:2.0:1782488055.799575:0:294639:0:(client.c:3739:ptlrpc_set_mbits()) resend with new mbits old x1869066294400128 new x1869066298631552 00000100:00100000:2.0:1782488055.799664:0:294639:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6aebfb3000 going to sleep for 464 seconds 00000100:00100000:1.0:1782488055.799996:0:2359:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2359:1869066298631168:192.168.202.121@tcp:38:kworker.0 00010000:00010000:0.0:1782488056.301921:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488057.332384:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488058.350358:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488059.374014:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488060.398238:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488060.846020:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:60894528-77cf-40cf-92ce-b54cc4bbae30:2362:1869066298631680:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488060.849894:0:2362:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1782488060.849918:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298631808:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488060.861427:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298632064:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488060.861527:0:2363:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1782488060.861535:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298631936:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488060.861708:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298631808:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488060.861744:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:60894528-77cf-40cf-92ce-b54cc4bbae30:2362:1869066298631680:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488060.877475:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298632064:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488060.878266:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298631936:192.168.202.121@tcp:400:kworker.0 00000080:00200000:3.0:1782488061.289376:0:320128:0:(file.c:6112:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08),name=/ 00000002:00010000:3.0:1782488061.289406:0:320128:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1782488061.289495:0:320128:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:3.0:1782488061.289513:0:320128:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00000080:00200000:3.0:1782488061.289522:0:320128:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1782488061.289538:0:320128:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) 00000080:00010000:3.0:1782488061.289542:0:320128:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) for lock 0x43f633357a48a726 00000080:00010000:3.0:1782488061.289552:0:320128:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a726 from it ffff99a4c151bbf8 00010000:00010000:3.0:1782488061.289556:0:320128:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 15 lru_type: 1 00010000:00010000:3.0:1782488061.289565:0:320128:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200000:3.0:1782488061.289583:0:320128:0:(file.c:6584:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), inode mode 41ff mask 1 00000080:00200000:3.0:1782488061.289597:0:320128:0:(file.c:6112:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08),name=/ 00000002:00010000:3.0:1782488061.289603:0:320128:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1782488061.289609:0:320128:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:3.0:1782488061.289617:0:320128:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00000080:00200000:3.0:1782488061.289623:0:320128:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1782488061.289626:0:320128:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) 00000080:00010000:3.0:1782488061.289629:0:320128:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) for lock 0x43f633357a48a726 00000080:00010000:3.0:1782488061.289632:0:320128:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a726 from it ffff99a4c151bb38 00010000:00010000:3.0:1782488061.289635:0:320128:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 16 lru_type: 1 00010000:00010000:3.0:1782488061.289641:0:320128:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00000080:00200000:3.0:1782488061.289659:0:320128:0:(file.c:6584:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), inode mode 41ff mask 3 00000080:00200000:3.0:1782488061.289663:0:320128:0:(namei.c:1412:ll_atomic_open()) VFS Op:name=f110.sanityn, dir=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), file ffff8a6ac93fb200, open_flags 8941, mode 81b6 opened 0 00000080:00200000:3.0:1782488061.289672:0:320128:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f110.sanityn, dir=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), intent=open|creat 00000002:00010000:3.0:1782488061.289690:0:320128:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f110.sanityn,[0x200000bd2:0xb22:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0104502 00010000:00010000:3.0:1782488061.289702:0:320128:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x8400000000 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:3.0:1782488061.289851:0:320128:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 1/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:3.0:1782488061.289859:0:320128:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1782488061.289862:0:320128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b017eae00/0x43f633357a48a726 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x4809400000020 nid: local remote: 0x66475b4e8b8c9c61 expref: -99 pid: 318425 timeout: 0 lvb_type: 0 lru_score: 17 lru_type: 1 00010000:00010000:3.0:1782488061.289896:0:320128:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 320128 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.289903:0:320128:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 320128 initiator: MDT0 lru_score: 0 lru_type: 0 00000020:00100000:3.0:1782488061.289914:0:320128:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8a6ac800b000: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1782488061.289926:0:320128:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8a6af52df480 x1869066298632320/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 768/66432 e 0 to 0 dl 0 ref 1 fl New:PQU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00010000:00010000:3.0:1782488061.289936:0:320128:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 320128 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1782488061.289958:0:320128:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52df480 pname:cluuid:pid:xid:nid:opc:job touch:5c4ca15f-f34e-4111-b105-de67ffe3ec92:320128:1869066298632320:192.168.202.121@tcp:101:touch.0 00000100:00100000:3.0:1782488061.290883:0:320128:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6ac4d73480 going to sleep for 710 seconds 00000100:00100000:0.0:1782488061.310135:0:320128:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52df480 pname:cluuid:pid:xid:nid:opc:job touch:5c4ca15f-f34e-4111-b105-de67ffe3ec92:320128:1869066298632320:192.168.202.121@tcp:101:touch.0 00010000:00010000:0.0:1782488061.310168:0:320128:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.310207:0:320128:0:(ldlm_request.c:774:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 4/0,1 mode: --/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.310217:0:320128:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 4/0,1 mode: --/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.310231:0:320128:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 4/0,1 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.310242:0:320128:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 4/0,1 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.310299:0:320128:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 5/1,1 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.310307:0:320128:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 5/1,1 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.310315:0:320128:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 4/1,0 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1782488061.310327:0:320128:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=3 disposition=37, status=0 req@ffff8a6af52df480 x1869066298632320/t8589967274(8589967274) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 768/664 e 0 to 0 dl 1782488776 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00100000:0.0:1782488061.310341:0:320128:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8a6af52df480 x1869066298632320/t8589967274(8589967274) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 768/664 e 0 to 0 dl 1782488776 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:0.0:1782488061.310354:0:320128:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:0.0:1782488061.310367:0:320128:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.310377:0:320128:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff8a6afafc7400 type 13 mode 4 res 8589937618/2850 (0 0) 00000080:00200000:0.0:1782488061.310384:0:320128:0:(namei.c:952:ll_intent_lock()) intent lock 3 on i1 [0x200000007:0x1:0x0] suppgids 0 -1: rc 0 00000080:00200000:0.0:1782488061.310400:0:320128:0:(namei.c:1623:ll_create_it()) VFS Op:name=f110.sanityn, dir=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), intent=open|creat 00000080:00200000:0.0:1782488061.310634:0:320128:0:(llite_lib.c:3226:ll_read_inode2()) VFS Op:inode=[0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448) 00000080:00200000:0.0:1782488061.310798:0:320128:0:(file.c:6749:ll_layout_conf()) [0x200000bd2:0xb22:0x0]: layout version change: 4294967294 -> 0 00000080:00010000:0.0:1782488061.310809:0:320128:0:(namei.c:1589:ll_create_node()) setting l_ast_data to inode [0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448) 00000080:00010000:0.0:1782488061.310819:0:320128:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448) for lock 0x43f633357a48a7f8 00000080:00010000:0.0:1782488061.310832:0:320128:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448) 00000080:00200008:0.0:1782488061.310848:0:320128:0:(file.c:1058:ll_file_open()) START file f110.sanityn:[0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448), flags 104501 00000002:00100000:0.0:1782488061.310871:0:320128:0:(mdc_request.c:811:mdc_set_open_replay_data()) @@@ Set up open replay data req@ffff8a6af52df480 x1869066298632320/t8589967274(8589967274) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/664 e 0 to 0 dl 1782488776 ref 2 fl Complete:RPQU/604/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000080:00010000:0.0:1782488061.310917:0:320128:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a7f8 from it ffff8a6acf5b6ea0 00010000:00010000:0.0:1782488061.310922:0:320128:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 3/1,0 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.310932:0:320128:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1782488061.310958:0:320128:0:(file.c:6584:ll_inode_permission()) VFS Op:inode=[0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448), inode mode 81a4 mask 40 00000080:00200008:0.0:1782488061.311013:0:320128:0:(llite_lib.c:2307:ll_setattr_raw()) START file f110.sanityn:[0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448) current size 0, valid attrs 20070, mode 8a6a, uid -1051607472, gid -26204, new size 18446744072200059415, atime 1782488061.000000000, mtime 1782488061.000000000, ctime 1782488061.000000000 00010000:00010000:0.0:1782488061.311043:0:320128:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 2/0,0 mode: PR/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x28400000000 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:0.0:1782488061.311069:0:320128:0:(vvp_object.c:117:vvp_conf_set()) [0x200000bd2:0xb22:0x0]: losing layout lock 00010000:00010000:0.0:1782488061.311110:0:320128:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 1/0,0 mode: --/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:0.0:1782488061.311116:0:320128:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1782488061.311119:0:320128:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031400/0x43f633357a48a7f8 lrc: 0/0,0 mode: --/PR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x4829400000020 nid: local remote: 0x66475b4e8b8c9dbf expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000020:00100000:0.0:1782488061.311137:0:320128:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8a6ac800b000: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:0.0:1782488061.311143:0:320128:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8a6af7f0a680 x1869066298632576/t0(0) o36->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488061.311166:0:320128:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af7f0a680 pname:cluuid:pid:xid:nid:opc:job touch:5c4ca15f-f34e-4111-b105-de67ffe3ec92:320128:1869066298632576:192.168.202.121@tcp:36:touch.0 00000100:00100000:0.0:1782488061.311246:0:320128:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6ac7c69900 going to sleep for 710 seconds 00000100:00100000:3.0:1782488061.321485:0:320128:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af7f0a680 pname:cluuid:pid:xid:nid:opc:job touch:5c4ca15f-f34e-4111-b105-de67ffe3ec92:320128:1869066298632576:192.168.202.121@tcp:36:touch.0 00000080:00200000:3.0:1782488061.321771:0:320128:0:(vvp_io.c:1822:vvp_io_init()) [0x200000bd2:0xb22:0x0] ignore/verify layout 0/0, layout version 0 restore needed 0 00000080:00010000:3.0:1782488061.321792:0:320128:0:(file.c:6971:ll_layout_intent()) ### lustre: requeue layout lock for file [0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448) 00000002:00010000:3.0:1782488061.321802:0:320128:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000bd2:0xb22:0x0]) in obj [0x200000bd2:0xb22:0x0], intent: layout flags 0 00010000:00010000:3.0:1782488061.321853:0:320128:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 3/1,0 mode: --/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 320128 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.321863:0:320128:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 3/1,0 mode: --/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 320128 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.321873:0:320128:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 3/1,0 mode: --/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x0 pid: 320128 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1782488061.321893:0:320128:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52de300 pname:cluuid:pid:xid:nid:opc:job touch:5c4ca15f-f34e-4111-b105-de67ffe3ec92:320128:1869066298632704:192.168.202.121@tcp:101:touch.0 00000100:00100000:3.0:1782488061.321949:0:320128:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6ac4d73240 going to sleep for 710 seconds 00000100:00100000:0.0:1782488061.323919:0:294639:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6adf730e00 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294639:1869066294400128:192.168.202.121@tcp:101:rm.500 00010000:00010000:0.0:1782488061.323940:0:294639:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.323953:0:294639:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.323960:0:294639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.323966:0:294639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.323975:0:294639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.323981:0:294639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8a6ac2c60600 already unused, calling callback (ffffffffc1b9bfa0) 00010000:00010000:0.0:1782488061.323985:0:294639:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.323991:0:294639:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.324037:0:294639:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 5/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.324044:0:294639:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1782488061.324066:0:294639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.324074:0:294639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac2c60600/0x43f633357a4784ec lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:0.0:1782488061.324099:0:294639:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=3, status=-116 req@ffff8a6adf730e00 x1869066294400128/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/584 e 3 to 0 dl 1782488524 ref 1 fl Complete:RQU/602/0 rc 301/301 job:'rm.500' uid:500 gid:500 projid:0 00000100:00100000:3.0:1782488061.342843:0:320128:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52de300 pname:cluuid:pid:xid:nid:opc:job touch:5c4ca15f-f34e-4111-b105-de67ffe3ec92:320128:1869066298632704:192.168.202.121@tcp:101:touch.0 00010000:00010000:3.0:1782488061.342878:0:320128:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 4/1,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.342893:0:320128:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 4/1,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1782488061.342907:0:320128:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=400 disposition=1, status=0 req@ffff8a6af52de300 x1869066298632704/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 376/400 e 0 to 0 dl 1782488776 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'touch.0' uid:0 gid:0 projid:0 00000002:00010000:3.0:1782488061.342918:0:320128:0:(mdc_locks.c:937:mdc_finish_enqueue()) ### layout lock returned by: layout, lvb_len: 56 ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 3/1,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1782488061.342949:0:320128:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 3/1,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.342962:0:320128:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff8a6afafc7400 type 13 mode 16 res 8589937618/2850 (0 0) 00000080:00200000:3.0:1782488061.342968:0:320128:0:(namei.c:952:ll_intent_lock()) intent lock 1024 on i1 [0x200000bd2:0xb22:0x0] suppgids 0 0: rc 0 00000080:00010000:3.0:1782488061.342987:0:320128:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448) for lock 0x43f633357a48a7ff 00000080:00010000:3.0:1782488061.342994:0:320128:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a7ff from it ffff99a4c151ba78 00010000:00010000:3.0:1782488061.342998:0:320128:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 3/1,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.343008:0:320128:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 2/0,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1782488061.343030:0:320128:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 2/1,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1782488061.343041:0:320128:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 2/1,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00010000:3.0:1782488061.343050:0:320128:0:(file.c:6854:ll_layout_lock_set()) ### file [0x200000bd2:0xb22:0x0](00000000f830c2c9) being reconfigured ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 3/1,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1782488061.343115:0:320128:0:(file.c:6749:ll_layout_conf()) [0x200000bd2:0xb22:0x0]: layout version change: 4294967294 -> 0 00010000:00010000:3.0:1782488061.343134:0:320128:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 3/1,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 1 lru_type: 0 00010000:00010000:3.0:1782488061.343153:0:320128:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0030200/0x43f633357a48a7ff lrc: 2/0,0 mode: CR/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0x66475b4e8b8c9ddb expref: -99 pid: 320128 timeout: 0 lvb_type: 3 lru_score: 2 lru_type: 0 00020000:00200000:3.0:1782488061.343172:0:320128:0:(lov_io.c:1086:lov_io_iter_init()) component[0] flags 0x10 00000100:00100000:2.0:1782488061.343369:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298632832:192.168.202.121@tcp:2:touch.0 00000100:00100000:2.0:1782488061.356865:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52de300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298632832:192.168.202.121@tcp:2:touch.0 00000080:00200000:3.0:1782488061.356985:0:320128:0:(vvp_io.c:304:vvp_io_fini()) [0x200000bd2:0xb22:0x0] ignore/verify layout 0/0, layout version 0 need write layout 0, restore needed 0, invalidate_lock 0 00000080:00200008:3.0:1782488061.357009:0:320128:0:(llite_lib.c:2557:ll_setattr_raw()) COMPLETED file f110.sanityn:[0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448) current size 0, valid attrs 100201f0, mode 8a6a, uid -1051607472, gid -26204, new size 18446744072200059415, atime 1782488061.000000000, mtime 1782488061.000000000, ctime 1782488061.000000000, rc 0 00000080:00200008:3.0:1782488061.357040:0:320128:0:(file.c:447:ll_file_release()) START file f110.sanityn:[0x200000bd2:0xb22:0x0](ffff8a6ad6b0f448), flags 104001 00000002:00100000:3.0:1782488061.357101:0:320128:0:(mdc_request.c:964:mdc_close()) @@@ matched open req@ffff8a6af52df480 x1869066298632320/t8589967274(8589967274) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/664 e 0 to 0 dl 1782488776 ref 1 fl Complete:RPQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000020:00100000:3.0:1782488061.357112:0:320128:0:(genops.c:2444:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff8a6ac800b000: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:3.0:1782488061.357117:0:320128:0:(client.c:806:ptlrpc_reassign_next_xid()) @@@ reassign xid req@ffff8a6af52dfb80 x1869066298633088/t0(0) o35->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:23/10 lens 392/736 e 0 to 0 dl 0 ref 1 fl New:QU/400/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:0 00000100:00100000:3.0:1782488061.357140:0:320128:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dfb80 pname:cluuid:pid:xid:nid:opc:job touch:5c4ca15f-f34e-4111-b105-de67ffe3ec92:320128:1869066298633088:192.168.202.121@tcp:35:touch.0 00000100:00100000:3.0:1782488061.357207:0:320128:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6ac4d73240 going to sleep for 11 seconds 00000100:00100000:1.0:1782488061.362277:0:320128:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dfb80 pname:cluuid:pid:xid:nid:opc:job touch:5c4ca15f-f34e-4111-b105-de67ffe3ec92:320128:1869066298633088:192.168.202.121@tcp:35:touch.0 00000002:00100000:1.0:1782488061.362597:0:320128:0:(mdc_request.c:832:mdc_free_open()) @@@ free open request, rq_replay=0 req@ffff8a6af52df480 x1869066298632320/t8589967274(8589967274) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/664 e 0 to 0 dl 1782488776 ref 1 fl Complete:RQU/604/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0 projid:0 00000080:00200000:0.0:1782488061.375246:0:294639:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x35:0x0] suppgids -1 -1: rc -116 00000080:00200000:0.0:1782488061.375325:0:294639:0:(file.c:6112:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08),name=/ 00000002:00010000:0.0:1782488061.375345:0:294639:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1782488061.375424:0:294639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.375435:0:294639:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488061.375444:0:294639:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1782488061.375468:0:294639:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af7f09500 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294639:1869066298633216:192.168.202.121@tcp:101:rm.500 00000080:00200000:3.0:1782488061.375527:0:294636:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f77r.sanityn-17, dir=[0x200000bd1:0x35:0x0](ffff8a6ad7484b08), flags=0 00000080:00200000:3.0:1782488061.375534:0:294636:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f77r.sanityn-17, dir=[0x200000bd1:0x35:0x0](ffff8a6ad7484b08), intent=getattr 00000002:00010000:3.0:1782488061.375549:0:294636:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f77r.sanityn-17,[0x0:0x0:0x0]) in obj [0x200000bd1:0x35:0x0], intent: getattr flags 01 00000100:00100000:0.0:1782488061.375553:0:294639:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6ac7c69540 going to sleep for 710 seconds 00010000:00010000:3.0:1782488061.375584:0:294636:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.375593:0:294636:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.375599:0:294636:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1782488061.375619:0:294636:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af52dd500 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294636:1869066298633344:192.168.202.121@tcp:101:rm.500 00000100:00100000:3.0:1782488061.375647:0:294636:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6ac4d73300 going to sleep for 710 seconds 00000100:00100000:3.0:1782488061.380352:0:294636:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af52dd500 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294636:1869066298633344:192.168.202.121@tcp:101:rm.500 00010000:00010000:3.0:1782488061.380374:0:294636:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380400:0:294636:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380408:0:294636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380416:0:294636:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380424:0:294636:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380430:0:294636:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8a6ac0031e00 already unused, calling callback (ffffffffc1b9bfa0) 00010000:00010000:3.0:1782488061.380435:0:294636:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380441:0:294636:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380448:0:294636:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 5/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380464:0:294636:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1782488061.380480:0:294636:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380487:0:294636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6ac0031e00/0x43f633357a48a80d lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 294636 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1782488061.380511:0:294636:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=3, status=-116 req@ffff8a6af52dd500 x1869066298633344/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/584 e 0 to 0 dl 1782488776 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'rm.500' uid:500 gid:500 projid:0 00000080:00200000:3.0:1782488061.380524:0:294636:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x35:0x0] suppgids -1 -1: rc -116 00000100:00100000:1.0:1782488061.380534:0:294639:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af7f09500 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294639:1869066298633216:192.168.202.121@tcp:101:rm.500 00010000:00010000:1.0:1782488061.380551:0:294639:0:(ldlm_request.c:759:ldlm_cli_enqueue_fini()) ### server returned different mode PR ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.380562:0:294639:0:(ldlm_request.c:787:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 4/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.380573:0:294639:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000080:00200000:3.0:1782488061.380576:0:294636:0:(file.c:6112:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08),name=/ 00010000:00010000:1.0:1782488061.380589:0:294639:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00010000:3.0:1782488061.380597:0:294636:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1782488061.380615:0:294636:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.380636:0:294636:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000080:00200000:3.0:1782488061.380643:0:294636:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids -1 -1: rc 1 00000080:00010000:3.0:1782488061.380649:0:294636:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) 00000080:00010000:3.0:1782488061.380653:0:294636:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) for lock 0x43f633357a48a806 00000080:00200000:2.0:1782488061.381354:0:294638:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f77r.sanityn-18, dir=[0x200000bd1:0x35:0x0](ffff8a6ad7484b08), flags=0 00000080:00200000:2.0:1782488061.381360:0:294638:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f77r.sanityn-18, dir=[0x200000bd1:0x35:0x0](ffff8a6ad7484b08), intent=getattr 00000002:00010000:2.0:1782488061.381377:0:294638:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f77r.sanityn-18,[0x0:0x0:0x0]) in obj [0x200000bd1:0x35:0x0], intent: getattr flags 01 00010000:00010000:2.0:1782488061.381655:0:294638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.381664:0:294638:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.381673:0:294638:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1782488061.381697:0:294638:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e95c00 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294638:1869066298633472:192.168.202.121@tcp:101:rm.500 00000100:00100000:2.0:1782488061.381975:0:294638:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6ad93aacc0 going to sleep for 710 seconds 00010000:00010000:1.0:1782488061.384584:0:294639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 7/3,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.384606:0:294639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 7/3,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.384614:0:294639:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 6/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1782488061.384633:0:294639:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=b, status=0 req@ffff8a6af7f09500 x1869066298633216/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 576/688 e 0 to 0 dl 1782488776 ref 1 fl Complete:RQU/600/0 rc 0/0 job:'rm.500' uid:500 gid:500 projid:0 00000002:00010000:1.0:1782488061.384647:0:294639:0:(mdc_locks.c:1357:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.384657:0:294639:0:(ldlm_lock.c:1539:ldlm_lock_match_with_skip()) ### not matched ns ffff8a6afafc7400 type 13 mode 4 res 8589934599/1 (0 0) 00000080:00200000:1.0:1782488061.384664:0:294639:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids -1 -1: rc 0 00000080:00200000:1.0:1782488061.384693:0:294639:0:(llite_lib.c:2989:ll_update_inode()) inode=[0x200000007:0x1:0x0], updating i_size 72192 00000080:00010000:1.0:1782488061.384698:0:294639:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) for lock 0x43f633357a48a806 00000080:00010000:1.0:1782488061.384703:0:294639:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) 00000080:00010000:1.0:1782488061.384705:0:294639:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a806 from it ffff99a4c8cdbb58 00010000:00010000:1.0:1782488061.384708:0:294639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 5/2,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.384715:0:294639:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 4/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000080:00200000:1.0:1782488061.385110:0:294639:0:(file.c:6584:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), inode mode 41ff mask 1 00000080:00200000:1.0:1782488061.385132:0:294639:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=d77r.sanityn, dir=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), flags=56 00000080:00200000:1.0:1782488061.385136:0:294639:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=d77r.sanityn, dir=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), intent=0 00000002:00010000:1.0:1782488061.385154:0:294639:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: d77r.sanityn,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1782488061.385195:0:294639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x10000000000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.385203:0:294639:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.385212:0:294639:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782488061.385233:0:294639:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6aef9b3100 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294639:1869066298633600:192.168.202.121@tcp:101:rm.500 00000100:00100000:1.0:1782488061.385571:0:294639:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6ac88f0b40 going to sleep for 710 seconds 00000080:00010000:3.0:1782488061.386604:0:294636:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a806 from it ffff99a4c8e9bc10 00010000:00010000:3.0:1782488061.386613:0:294636:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.386626:0:294636:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00200000:3.0:1782488061.386642:0:294636:0:(file.c:6584:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), inode mode 41ff mask 1 00000100:00100000:2.0:1782488061.396222:0:294639:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6aef9b3100 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294639:1869066298633600:192.168.202.121@tcp:101:rm.500 00010000:00010000:2.0:1782488061.396244:0:294639:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x0 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396258:0:294639:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396266:0:294639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396273:0:294639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x10206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396294:0:294639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396301:0:294639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8a6af7f42600 already unused, calling callback (ffffffffc1b9bfa0) 00010000:00010000:2.0:1782488061.396307:0:294639:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396315:0:294639:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x206400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396323:0:294639:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x20e400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396331:0:294639:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1782488061.396338:0:294639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782488061.396345:0:294638:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e95c00 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294638:1869066298633472:192.168.202.121@tcp:101:rm.500 00010000:00010000:2.0:1782488061.396362:0:294639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42600/0x43f633357a48a81b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT flags: 0x4a0f400000000 pid: 294639 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.396363:0:294638:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.396397:0:294638:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:2.0:1782488061.396399:0:294639:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=10 disposition=7, status=0 req@ffff8a6aef9b3100 x1869066298633600/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/584 e 0 to 0 dl 1782488776 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'rm.500' uid:500 gid:500 projid:0 00000080:00200000:2.0:1782488061.396418:0:294639:0:(namei.c:952:ll_intent_lock()) intent lock 16 on i1 [0x200000007:0x1:0x0] suppgids -1 -1: rc 0 00010000:00010000:2.0:1782488061.396463:0:294639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1782488061.396476:0:294639:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00000080:00010000:2.0:1782488061.396485:0:294639:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a806 from it ffff99a4c8cdb8d0 00010000:00010000:2.0:1782488061.396489:0:294639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 1 lru_type: 0 00010000:00010000:2.0:1782488061.396497:0:294639:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00000080:00200000:2.0:1782488061.396575:0:294639:0:(file.c:6112:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08),name=/ 00000002:00010000:2.0:1782488061.396589:0:294639:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1782488061.396596:0:294639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1782488061.396607:0:294638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396610:0:294639:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1782488061.396625:0:294638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1782488061.396641:0:294639:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids -1 -1: rc 1 00000080:00010000:2.0:1782488061.396651:0:294639:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) 00010000:00010000:1.0:1782488061.396654:0:294638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00010000:2.0:1782488061.396656:0:294639:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) for lock 0x43f633357a48a806 00000080:00010000:2.0:1782488061.396668:0:294639:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a806 from it ffff99a4c8cdbc10 00010000:00010000:1.0:1782488061.396668:0:294638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8a6b0055e200 already unused, calling callback (ffffffffc1b9bfa0) 00010000:00010000:1.0:1782488061.396675:0:294638:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396676:0:294639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 2 lru_type: 0 00010000:00010000:1.0:1782488061.396692:0:294638:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488061.396696:0:294639:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1782488061.396702:0:294638:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 5/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1782488061.396709:0:294639:0:(file.c:6584:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), inode mode 41ff mask 1 00010000:00010000:1.0:1782488061.396710:0:294638:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1782488061.396716:0:294638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00000080:00200000:2.0:1782488061.396935:0:294639:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d77r.sanityn, flags=24 00010000:00010000:1.0:1782488061.397115:0:294638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6b0055e200/0x43f633357a48a814 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 294638 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:1.0:1782488061.397138:0:294638:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=3, status=-116 req@ffff8a6b01e95c00 x1869066298633472/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/584 e 0 to 0 dl 1782488776 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'rm.500' uid:500 gid:500 projid:0 00000080:00200000:1.0:1782488061.397148:0:294638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x35:0x0] suppgids -1 -1: rc -116 00000080:00200000:1.0:1782488061.397184:0:294638:0:(file.c:6112:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08),name=/ 00000002:00010000:1.0:1782488061.397195:0:294638:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1782488061.397203:0:294638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1782488061.397213:0:294638:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00000080:00200000:1.0:1782488061.397227:0:294638:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids -1 -1: rc 1 00000080:00010000:1.0:1782488061.397232:0:294638:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) 00000080:00010000:1.0:1782488061.397235:0:294638:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) for lock 0x43f633357a48a806 00000080:00010000:1.0:1782488061.397240:0:294638:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a806 from it ffff99a4c8debc10 00010000:00010000:1.0:1782488061.397244:0:294638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 3 lru_type: 0 00010000:00010000:1.0:1782488061.397253:0:294638:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:1.0:1782488061.397276:0:294638:0:(file.c:6584:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), inode mode 41ff mask 1 00000080:00200000:2.0:1782488061.397444:0:294636:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d77r.sanityn, flags=56 00000080:00200000:1.0:1782488061.398851:0:294638:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d77r.sanityn, flags=56 00000080:00200000:1.0:1782488061.400874:0:294637:0:(namei.c:1335:ll_lookup_nd()) VFS Op:name=f77r.sanityn-19, dir=[0x200000bd1:0x35:0x0](ffff8a6ad7484b08), flags=0 00000080:00200000:1.0:1782488061.400887:0:294637:0:(namei.c:1072:ll_lookup_it()) VFS Op:name=f77r.sanityn-19, dir=[0x200000bd1:0x35:0x0](ffff8a6ad7484b08), intent=getattr 00000002:00010000:1.0:1782488061.400916:0:294637:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: f77r.sanityn-19,[0x0:0x0:0x0]) in obj [0x200000bd1:0x35:0x0], intent: getattr flags 01 00010000:00010000:1.0:1782488061.400976:0:294637:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x10000000000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.400988:0:294637:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488061.400997:0:294637:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 3/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1782488061.401361:0:294637:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6aef9b3800 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294637:1869066298633728:192.168.202.121@tcp:101:rm.500 00000100:00100000:1.0:1782488061.401645:0:294637:0:(client.c:2701:ptlrpc_set_wait()) set ffff8a6ac88f0540 going to sleep for 710 seconds 00000100:00100000:3.0:1782488061.418713:0:294637:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6aef9b3800 pname:cluuid:pid:xid:nid:opc:job rm:5c4ca15f-f34e-4111-b105-de67ffe3ec92:294637:1869066298633728:192.168.202.121@tcp:101:rm.500 00010000:00010000:3.0:1782488061.418910:0:294637:0:(ldlm_request.c:688:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x0 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.418925:0:294637:0:(ldlm_request.c:591:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.418933:0:294637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 4/1,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.418941:0:294637:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x10206400000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.418950:0:294637:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x206400000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.418956:0:294637:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff8a6af7f42800 already unused, calling callback (ffffffffc1b9bfa0) 00010000:00010000:3.0:1782488061.418960:0:294637:0:(ldlm_request.c:1732:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.418967:0:294637:0:(ldlm_request.c:1742:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 4/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x206400000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.418975:0:294637:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 5/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x20e400000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.418982:0:294637:0:(ldlm_request.c:1407:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1782488061.418988:0:294637:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 3/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1782488061.418995:0:294637:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6af7f42800/0x43f633357a48a822 lrc: 0/0,0 mode: --/CR res: [0x200000bd1:0x35:0x0].0x0 bits 0x2/0x2 rrc: 2 type: IBT flags: 0x4a0f400000000 pid: 294637 initiator: MDT0 lru_score: 0 lru_type: 0 00000002:00100000:3.0:1782488061.419034:0:294637:0:(mdc_locks.c:833:mdc_finish_enqueue()) @@@ op=8 disposition=3, status=-116 req@ffff8a6aef9b3800 x1869066298633728/t0(0) o101->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 584/584 e 0 to 0 dl 1782488776 ref 1 fl Complete:RQU/600/0 rc 301/301 job:'rm.500' uid:500 gid:500 projid:0 00000080:00200000:3.0:1782488061.419049:0:294637:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000bd1:0x35:0x0] suppgids -1 -1: rc -116 00000080:00200000:3.0:1782488061.419105:0:294637:0:(file.c:6112:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08),name=/ 00000002:00010000:3.0:1782488061.419122:0:294637:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1782488061.419134:0:294637:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1782488061.419146:0:294637:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00000080:00200000:3.0:1782488061.419152:0:294637:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids -1 -1: rc 1 00000080:00010000:3.0:1782488061.419159:0:294637:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) 00000080:00010000:3.0:1782488061.419162:0:294637:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) for lock 0x43f633357a48a806 00000080:00010000:3.0:1782488061.419166:0:294637:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a806 from it ffff99a4c8dc3c10 00010000:00010000:3.0:1782488061.419169:0:294637:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 4 lru_type: 0 00010000:00010000:3.0:1782488061.419176:0:294637:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1782488061.419189:0:294637:0:(file.c:6584:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08), inode mode 41ff mask 1 00000080:00200000:3.0:1782488061.419196:0:294637:0:(dcache.c:276:ll_revalidate_dentry()) VFS Op:name=d77r.sanityn, flags=56 00010000:00010000:0.0:1782488061.427514:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200008:3.0:1782488061.460243:0:320134:0:(file.c:6217:ll_getattr_dentry()) START file /:[0x200000007:0x1:0x0](ffff8a6ad6b32a08), request_mask 0, flags 0, foreign 0 00000080:00200000:3.0:1782488061.460254:0:320134:0:(file.c:6112:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](ffff8a6ad6b32a08),name=/ 00000002:00010000:3.0:1782488061.460277:0:320134:0:(mdc_locks.c:1506:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1782488061.460316:0:320134:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1782488061.460333:0:320134:0:(ldlm_lock.c:1522:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00000080:00200000:3.0:1782488061.460342:0:320134:0:(namei.c:952:ll_intent_lock()) intent lock 8 on i1 [0x200000007:0x1:0x0] suppgids 0 0: rc 1 00000080:00010000:3.0:1782488061.460350:0:320134:0:(dcache.c:255:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) 00000080:00010000:3.0:1782488061.460354:0:320134:0:(llite_internal.h:2008:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](ffff8a6ad6b32a08) for lock 0x43f633357a48a806 00000080:00010000:3.0:1782488061.460362:0:320134:0:(dcache.c:152:ll_intent_drop_lock()) releasing lock with cookie 0x43f633357a48a806 from it ffff99a4c8debcb8 00010000:00010000:3.0:1782488061.460366:0:320134:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 5 lru_type: 0 00010000:00010000:3.0:1782488061.460375:0:320134:0:(ldlm_lock.c:912:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff8a6ac800b000 lock: ffff8a6aee037000/0x43f633357a48a806 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x66475b4e8b8c9de9 expref: -99 pid: 294639 timeout: 0 lvb_type: 0 lru_score: 6 lru_type: 0 00000100:00000400:1.0:1782488062.383977:0:2360:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1782487493/real 1782487493] req@ffff8a6b00487480 x1869066284548096/t0(0) o400->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 224/224 e 22 to 1 dl 1782488062 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1782488062.446742:0:2360:0:(client.c:2533:ptlrpc_expire_one_request()) @@@ TIMEOUT-NORESEND: req@ffff8a6b00487480 x1869066284548096/t0(0) o400->lustre-MDT0000-mdc-ffff8a6ac800b000@192.168.202.121@tcp:12/10 lens 224/224 e 22 to 1 dl 1782488062 ref 1 fl Rpc:XNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1782488062.446769:0:2360:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b00487480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2360:1869066284548096:192.168.202.121@tcp:400:kworker.0 00010000:00010000:1.0:1782488062.446952:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488063.470031:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488064.507854:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488065.518260:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488065.972387:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298633856:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488065.972478:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298634112:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488065.972511:0:2363:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1782488065.972518:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298633984:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488065.972542:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e94380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298634240:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488065.989872:0:2363:0:(client.c:3111:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8a6ac800b000: committing for last_committed 8589967276 gen 1 00000100:00100000:2.0:1782488065.989912:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298633984:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488065.989925:0:2363:0:(client.c:3111:ptlrpc_free_committed()) lustre-OST0001-osc-ffff8a6ac800b000: committing for last_committed 8589941508 gen 4 00000100:00100000:2.0:1782488065.989931:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e94380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298634240:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488065.989959:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298633856:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488065.989969:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298634112:192.168.202.121@tcp:400:kworker.0 00010000:00010000:3.0:1782488066.544442:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488067.570148:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488068.591124:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488069.617206:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488070.638860:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1782488071.046556:0:320703:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 11:34:29 (1782488069) 00000100:00100000:2.0:1782488071.086529:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298634496:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488071.086686:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298634752:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488071.086735:0:2362:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1782488071.086742:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e97480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:60894528-77cf-40cf-92ce-b54cc4bbae30:2362:1869066298634368:192.168.202.121@tcp:400:kworker.0 00000100:00100000:2.0:1782488071.086768:0:2362:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e94380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298634624:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488071.098473:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:60894528-77cf-40cf-92ce-b54cc4bbae30:2362:1869066298634368:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488071.098497:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e94380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298634624:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488071.098508:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298634496:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488071.098517:0:2362:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e97100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2362:1869066298634752:192.168.202.121@tcp:400:kworker.0 00010000:00010000:3.0:1782488071.662706:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488072.700793:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1782488073.705373:0:320839:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanityn test_111 needs >= 2 MDTs 00010000:00010000:2.0:1782488073.711328:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488074.733956:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1782488075.682996:0:320992:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 112: update max-inherit in default LMV === 11:34:34 (1782488074) 00010000:00010000:3.0:1782488075.761216:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488076.206767:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298634880:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488076.206851:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e94a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298635136:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488076.206878:0:2363:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1782488076.206884:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e94e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298635008:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488076.206902:0:2363:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6b01e96680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298635264:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488076.210893:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e94e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298635008:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488076.210907:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e96680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298635264:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488076.210917:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e95880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2363:1869066298634880:192.168.202.121@tcp:400:kworker.0 00000100:00100000:3.0:1782488076.210926:0:2363:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6b01e94a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2363:1869066298635136:192.168.202.121@tcp:400:kworker.0 00010000:00010000:0.0:1782488076.783240:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1782488077.353537:0:321128:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanityn test_112 We need at least 2 MDTs for this test 00010000:00010000:2.0:1782488077.807940:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488078.839814:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488079.854005:0:265367:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1782488080.076050:0:321280:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 113: check servers of specified fs ======= 11:34:38 (1782488078) 00010000:00010000:0.0:1782488080.880943:0:261832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488081.326428:0:2361:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af7f09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2361:1869066298635520:192.168.202.121@tcp:400:kworker.0 00000100:00100000:0.0:1782488081.326536:0:2361:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af7f0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2361:1869066298635776:192.168.202.121@tcp:400:kworker.0 00000100:00100000:0.0:1782488081.326568:0:2361:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1782488081.326575:0:2361:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af7f0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2361:1869066298635392:192.168.202.121@tcp:400:kworker.0 00000100:00100000:0.0:1782488081.326601:0:2361:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff8a6af7f08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2361:1869066298635648:192.168.202.121@tcp:400:kworker.0 00000100:00100000:0.0:1782488081.352918:0:2361:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af7f0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:60894528-77cf-40cf-92ce-b54cc4bbae30:2361:1869066298635392:192.168.202.121@tcp:400:kworker.0 00000100:00100000:0.0:1782488081.352943:0:2361:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af7f08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2361:1869066298635648:192.168.202.121@tcp:400:kworker.0 00000100:00100000:0.0:1782488081.352956:0:2361:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af7f09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2361:1869066298635520:192.168.202.121@tcp:400:kworker.0 00000100:00100000:0.0:1782488081.352968:0:2361:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff8a6af7f0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5c4ca15f-f34e-4111-b105-de67ffe3ec92:2361:1869066298635776:192.168.202.121@tcp:400:kworker.0 00010000:00010000:1.0:1782488081.902964:0:261170:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1782488082.135946:0:321421:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: sanityn test_113: @@@@@@ FAIL: cannot get instance of /mnt/lustre 00010000:00010000:1.0:1782488082.928105:0:261169:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488083.960673:0:265368:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 525 lines, 525 kept, 0 dropped, 0 bad.