00010000:00010000:2.0F:1788972460.843647:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0F:1788972461.870055:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788972462.895571:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0F:1788972463.170840:0:57410:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.24@tcp (source: 12345-192.168.203.24@tcp) 00000100:00100000:1.0:1788972463.170909:0:57410:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.24@tcp (source: 12345-192.168.203.24@tcp) 00000100:00100000:0.0F:1788972463.171006:0:61016:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872179097344 00000100:00100000:0.0:1788972463.171021:0:61016:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872179097344 00000100:00100000:0.0:1788972463.171100:0:61016:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.24@tcp, seq: 1012 00000100:00100000:0.0:1788972463.171110:0:61016:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b523100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:bb578dba-66a2-4424-b6c3-0338dda03657+6:100241:x1875872179097344:12345-192.168.203.24@tcp:400:kworker.0 00000100:00100000:0.0:1788972463.171174:0:61016:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b523100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:bb578dba-66a2-4424-b6c3-0338dda03657+6:100241:x1875872179097344:12345-192.168.203.24@tcp:400:kworker.0 Request processed in 66us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972463.171181:0:61016:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.24@tcp, seq: 1012 00000100:00080000:1.0:1788972463.171395:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788972463.171595:0:58107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872179097216 00000100:00100000:3.0:1788972463.171600:0:58107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872179097216 00000100:00100000:3.0:1788972463.171637:0:58107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.24@tcp, seq: 601 00000100:00100000:3.0:1788972463.171643:0:58107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1a8c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:94d249cb-8ce8-4694-9530-465f9f595f02+10:100241:x1875872179097216:12345-192.168.203.24@tcp:400:kworker.0 00000100:00100000:3.0:1788972463.171683:0:58107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1a8c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:94d249cb-8ce8-4694-9530-465f9f595f02+10:100241:x1875872179097216:12345-192.168.203.24@tcp:400:kworker.0 Request processed in 40us (774us total) trans 0 rc 0/0 00000100:00100000:3.0:1788972463.171690:0:58107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.24@tcp, seq: 601 00010000:00010000:3.0:1788972463.915875:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1788972463.984217:0:57419:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1788972463.984239:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928ababb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:57419:1875872192904192:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788972463.984305:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788972463.984970:0:58919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192904192 00000100:00100000:3.0:1788972463.984978:0:58919:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192904192 00000100:00100000:3.0:1788972463.985046:0:58919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:3.0:1788972463.985055:0:58919:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928abab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57419:x1875872192904192:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1788972463.985087:0:58919:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00080000:1.0:1788972463.985125:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788972463.985139:0:58919:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928abab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57419:x1875872192904192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (833us total) trans 0 rc 0/0 00000100:00100000:3.0:1788972463.985146:0:58919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00010000:00080000:3.0:1788972463.985183:0:57419:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928ababb80 x1875872192904192/t0(0) o13->lustre-OST0000-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1788972479 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788972463.985201:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928ababb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:57419:1875872192904192:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1788972464.493392:0:57418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928b56d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:57418:1875872192904448:0@lo:400:kworker.0 00000100:00100000:1.0:1788972464.493559:0:57417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928ababb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17:57417:1875872192904320:0@lo:400:kworker.0 00000100:00100000:1.0:1788972464.493596:0:57417:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788972464.493639:0:57417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928b56e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:57417:1875872192904832:0@lo:400:kworker.0 00000100:00100000:1.0:1788972464.493663:0:57417:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788972464.493722:0:58110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192904320 00000100:00100000:1.0:1788972464.493729:0:58110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192904320 00000100:00100000:1.0:1788972464.493774:0:58110:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 602 00000100:00100000:1.0:1788972464.493782:0:58110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1a8d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17+19:57417:x1875872192904320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788972464.493831:0:58110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1a8d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17+19:57417:x1875872192904320:12345-0@lo:400:kworker.0 Request processed in 49us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972464.493838:0:58110:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 602 00000100:00100000:1.0:1788972464.493867:0:58117:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192904832 00000100:00100000:1.0:1788972464.493871:0:58117:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192904832 00000100:00100000:1.0:1788972464.493905:0:58117:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1013 00000100:00100000:1.0:1788972464.493911:0:58117:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b521880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:57417:x1875872192904832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788972464.493942:0:58117:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b521880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:57417:x1875872192904832:12345-0@lo:400:kworker.0 Request processed in 31us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972464.493947:0:58117:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1013 00000100:00080000:1.0:1788972464.493965:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788972464.493990:0:57417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928ababb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17:57417:1875872192904320:0@lo:400:kworker.0 00000100:00100000:1.0:1788972464.494003:0:57417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928b56e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:57417:1875872192904832:0@lo:400:kworker.0 00000100:00100000:0.0:1788972464.494048:0:57418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972464.494088:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928b56fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:57419:1875872192904576:0@lo:400:kworker.0 00000100:00100000:0.0:1788972464.494113:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972464.494131:0:57419:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1788972464.494137:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928b56ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:57419:1875872192904704:0@lo:400:kworker.0 00000100:00100000:0.0:1788972464.494158:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972464.494183:0:61016:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192904576 00000100:00100000:0.0:1788972464.494187:0:61016:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192904576 00000100:00100000:0.0:1788972464.494207:0:61016:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192904704 00000100:00100000:0.0:1788972464.494209:0:61016:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192904704 00000100:00100000:0.0:1788972464.494226:0:61016:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1014 00000100:00100000:0.0:1788972464.494232:0:61016:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b56df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:57419:x1875872192904576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788972464.494265:0:61016:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b56df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:57419:x1875872192904576:12345-0@lo:400:kworker.0 Request processed in 33us (151us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972464.494272:0:61016:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1014 00000100:00100000:0.0:1788972464.494287:0:61016:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1015 00000100:00100000:0.0:1788972464.494304:0:61016:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b56e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:57419:x1875872192904704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788972464.494330:0:61016:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b56e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:57419:x1875872192904704:12345-0@lo:400:kworker.0 Request processed in 27us (172us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972464.494336:0:61016:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1015 00000100:00100000:0.0:1788972464.494360:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928b56ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:57419:1875872192904704:0@lo:400:kworker.0 00000100:00100000:0.0:1788972464.494368:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928b56fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:57419:1875872192904576:0@lo:400:kworker.0 00000100:00100000:1.0:1788972464.494441:0:58915:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192904448 00000100:00100000:1.0:1788972464.494723:0:58915:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192904448 00000100:00100000:1.0:1788972464.494759:0:58915:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:1.0:1788972464.494766:0:58915:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b56ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:57418:x1875872192904448:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788972464.494805:0:58915:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b56ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:57418:x1875872192904448:12345-0@lo:400:kworker.0 Request processed in 40us (1324us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972464.494811:0:58915:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00010000:00080000:1.0:1788972464.494839:0:57418:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928b56d880 x1875872192904448/t0(0) o400->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 0 dl 1788972480 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788972464.494849:0:57418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928b56d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:57418:1875872192904448:0@lo:400:kworker.0 00000100:00080000:1.0:1788972464.494864:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1788972464.494868:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1788972464.495568:0:57417:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788972464.495580:0:57417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928689e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192904960:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1788972464.495622:0:57417:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788972464.495658:0:61438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192904960 00000100:00100000:2.0:1788972464.495663:0:61438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192904960 00000100:00100000:2.0:1788972464.495708:0:61438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:2.0:1788972464.495716:0:61438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928689f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192904960:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1788972464.495743:0:61438:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788972464.495780:0:61438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928689f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192904960:12345-0@lo:13:osp-pre-1-0.0 Request processed in 66us (157us total) trans 0 rc 0/0 00000100:00100000:2.0:1788972464.495790:0:61438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00010000:00080000:2.0:1788972464.495825:0:57417:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928689e680 x1875872192904960/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1788972480 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788972464.495838:0:57417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928689e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192904960:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1788972464.941509:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788972465.962851:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788972466.991885:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788972468.011584:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788972468.287351:0:57410:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.24@tcp (source: 12345-192.168.203.24@tcp) 00000100:00100000:3.0:1788972468.287397:0:57410:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.24@tcp (source: 12345-192.168.203.24@tcp) 00000100:00100000:3.0:1788972468.287428:0:59755:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872179097600 00000100:00100000:3.0:1788972468.287438:0:59755:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872179097600 00000100:00100000:0.0:1788972468.287553:0:58109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872179097472 00000100:00100000:0.0:1788972468.287557:0:58109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872179097472 00000100:00100000:0.0:1788972468.287607:0:58109:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.24@tcp, seq: 603 00000100:00100000:0.0:1788972468.287616:0:58109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1a8b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:94d249cb-8ce8-4694-9530-465f9f595f02+10:100240:x1875872179097472:12345-192.168.203.24@tcp:400:kworker.0 00000100:00100000:0.0:1788972468.287752:0:58109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1a8b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:94d249cb-8ce8-4694-9530-465f9f595f02+10:100240:x1875872179097472:12345-192.168.203.24@tcp:400:kworker.0 Request processed in 135us (409us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972468.287760:0:58109:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.24@tcp, seq: 603 00000100:00100000:3.0:1788972468.287876:0:59755:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.24@tcp, seq: 1016 00000100:00100000:3.0:1788972468.287883:0:59755:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928a635500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:bb578dba-66a2-4424-b6c3-0338dda03657+6:100240:x1875872179097600:12345-192.168.203.24@tcp:400:kworker.0 00000100:00100000:3.0:1788972468.287925:0:59755:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928a635500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:bb578dba-66a2-4424-b6c3-0338dda03657+6:100240:x1875872179097600:12345-192.168.203.24@tcp:400:kworker.0 Request processed in 43us (528us total) trans 0 rc 0/0 00000100:00100000:3.0:1788972468.287932:0:59755:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.24@tcp, seq: 1016 00000100:00080000:0.0:1788972468.287985:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788972469.037161:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788972469.098673:0:57420:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1788972469.098697:0:57420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928b56fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:57420:1875872192905088:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788972469.098766:0:57420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972469.098890:0:58918:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192905088 00000100:00100000:0.0:1788972469.098898:0:58918:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192905088 00000100:00100000:0.0:1788972469.098961:0:58918:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:0.0:1788972469.098971:0:58918:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928a634700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57420:x1875872192905088:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1788972469.099005:0:58918:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1788972469.099065:0:58918:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928a634700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57420:x1875872192905088:12345-0@lo:13:osp-pre-0-0.0 Request processed in 96us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972469.099073:0:58918:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00010000:00080000:0.0:1788972469.099113:0:57420:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928b56fb80 x1875872192905088/t0(0) o13->lustre-OST0000-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1788972485 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788972469.099132:0:57420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928b56fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:57420:1875872192905088:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1788972469.099364:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1788972469.611693:0:57417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928689df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192905344:0@lo:400:kworker.0 00000100:00100000:2.0:1788972469.611771:0:57417:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788972469.611847:0:57417:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1788972469.611854:0:57417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9793c162a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:57417:1875872192905472:0@lo:400:kworker.0 00000100:00100000:2.0:1788972469.611880:0:57417:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788972469.611916:0:58917:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192905344 00000100:00100000:2.0:1788972469.611924:0:58917:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192905344 00000100:00100000:2.0:1788972469.611979:0:58917:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1788972469.612004:0:58917:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192905344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788972469.612058:0:58917:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1628380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192905344:12345-0@lo:400:kworker.0 Request processed in 57us (287us total) trans 0 rc 0/0 00000100:00100000:2.0:1788972469.612076:0:58917:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1788972469.612102:0:58116:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192905472 00000100:00100000:2.0:1788972469.612105:0:58116:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192905472 00000100:00100000:2.0:1788972469.612130:0:58116:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1017 00000100:00100000:2.0:1788972469.612136:0:58116:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1628000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:57417:x1875872192905472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788972469.612166:0:58116:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1628000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:57417:x1875872192905472:12345-0@lo:400:kworker.0 Request processed in 30us (285us total) trans 0 rc 0/0 00000100:00100000:2.0:1788972469.612172:0:58116:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1017 00000100:00080000:2.0:1788972469.612189:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1788972469.612221:0:57417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9793c162a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:57417:1875872192905472:0@lo:400:kworker.0 00010000:00080000:2.0:1788972469.612235:0:57417:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928689df80 x1875872192905344/t0(0) o400->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 0 dl 1788972485 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788972469.612245:0:57417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928689df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192905344:0@lo:400:kworker.0 00000100:00100000:1.0:1788972469.612331:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9793c162a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:57419:1875872192905600:0@lo:400:kworker.0 00000100:00100000:1.0:1788972469.612363:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788972469.612392:0:57419:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1788972469.612399:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928689e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17:57419:1875872192905216:0@lo:400:kworker.0 00000100:00100000:1.0:1788972469.612426:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788972469.612470:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9793c162b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:57419:1875872192905728:0@lo:400:kworker.0 00000100:00100000:1.0:1788972469.612495:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788972469.612519:0:58115:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192905600 00000100:00100000:1.0:1788972469.612523:0:58115:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192905600 00000100:00100000:1.0:1788972469.612552:0:58115:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192905728 00000100:00100000:1.0:1788972469.612555:0:58115:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192905728 00000100:00100000:1.0:1788972469.612576:0:58115:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1018 00000100:00100000:1.0:1788972469.612583:0:58115:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b523800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:57419:x1875872192905600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788972469.612619:0:58115:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b523800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:57419:x1875872192905600:12345-0@lo:400:kworker.0 Request processed in 37us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972469.612636:0:58115:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1018 00000100:00100000:1.0:1788972469.612649:0:58115:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1019 00000100:00100000:1.0:1788972469.612666:0:58115:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b523b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:57419:x1875872192905728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788972469.612696:0:58115:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b523b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:57419:x1875872192905728:12345-0@lo:400:kworker.0 Request processed in 42us (201us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972469.612702:0:58115:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1019 00000100:00100000:1.0:1788972469.612720:0:58107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192905216 00000100:00100000:1.0:1788972469.612724:0:58107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192905216 00000100:00100000:1.0:1788972469.612748:0:58107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 604 00000100:00100000:1.0:1788972469.612754:0:58107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1a88c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17+19:57419:x1875872192905216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788972469.612783:0:58107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1a88c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17+19:57419:x1875872192905216:12345-0@lo:400:kworker.0 Request processed in 29us (356us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972469.612789:0:58107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 604 00000100:00080000:1.0:1788972469.612814:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1788972469.612832:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928689e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17:57419:1875872192905216:0@lo:400:kworker.0 00000100:00100000:1.0:1788972469.612842:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9793c162b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:57419:1875872192905728:0@lo:400:kworker.0 00000100:00100000:1.0:1788972469.612849:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9793c162a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:57419:1875872192905600:0@lo:400:kworker.0 00000100:00100000:3.0:1788972469.613184:0:57418:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1788972469.613194:0:57418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928a635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:57418:1875872192905856:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788972469.613228:0:57418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1788972469.613260:0:58919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192905856 00000100:00100000:3.0:1788972469.613264:0:58919:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192905856 00000100:00100000:3.0:1788972469.613292:0:58919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:3.0:1788972469.613298:0:58919:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928a636d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57418:x1875872192905856:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1788972469.613324:0:58919:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1788972469.613359:0:58919:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928a636d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57418:x1875872192905856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 61us (130us total) trans 0 rc 0/0 00000100:00100000:3.0:1788972469.613365:0:58919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00010000:00080000:3.0:1788972469.613395:0:57418:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928a635880 x1875872192905856/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1788972485 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788972469.613409:0:57418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928a635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:57418:1875872192905856:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1788972470.059708:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1788972471.082829:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1788972472.087154:0:75545:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 12:47:50 (1788972470) 00010000:00010000:1.0:1788972472.107779:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788972473.132379:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788972473.409873:0:57411:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.24@tcp (source: 12345-192.168.203.24@tcp) 00000100:00100000:1.0:1788972473.409962:0:57411:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.24@tcp (source: 12345-192.168.203.24@tcp) 00000100:00100000:0.0:1788972473.411811:0:58115:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872179097856 00000100:00100000:0.0:1788972473.411826:0:58115:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872179097856 00000100:00100000:0.0:1788972473.411896:0:58115:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.24@tcp, seq: 1020 00000100:00100000:0.0:1788972473.411907:0:58115:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928ababb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:bb578dba-66a2-4424-b6c3-0338dda03657+6:100243:x1875872179097856:12345-192.168.203.24@tcp:400:kworker.0 00000100:00100000:0.0:1788972473.411984:0:58115:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928ababb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:bb578dba-66a2-4424-b6c3-0338dda03657+6:100243:x1875872179097856:12345-192.168.203.24@tcp:400:kworker.0 Request processed in 79us (2021us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972473.411992:0:58115:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.24@tcp, seq: 1020 00000100:00080000:1.0:1788972473.412162:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788972473.418502:0:58110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872179097728 00000100:00100000:3.0:1788972473.418517:0:58110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872179097728 00000100:00100000:3.0:1788972473.418641:0:58110:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.24@tcp, seq: 605 00000100:00100000:3.0:1788972473.418651:0:58110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1a88850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:94d249cb-8ce8-4694-9530-465f9f595f02+10:100242:x1875872179097728:12345-192.168.203.24@tcp:400:kworker.0 00000100:00100000:3.0:1788972473.418744:0:58110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1a88850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:94d249cb-8ce8-4694-9530-465f9f595f02+10:100242:x1875872179097728:12345-192.168.203.24@tcp:400:kworker.0 Request processed in 93us (8881us total) trans 0 rc 0/0 00000100:00100000:3.0:1788972473.418752:0:58110:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.24@tcp, seq: 605 00000001:02000400:0.0:1788972473.765186:0:75593:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type 00010000:00010000:1.0:1788972474.160100:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1788972474.220476:0:57418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928b56fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:57418:1875872192905984:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1788972474.220542:0:57418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788972474.220725:0:61438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192905984 00000100:00100000:2.0:1788972474.220732:0:61438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192905984 00000100:00100000:2.0:1788972474.220789:0:61438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 510 00000100:00100000:2.0:1788972474.220798:0:61438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928a635880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57418:x1875872192905984:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1788972474.220826:0:61438:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788972474.220864:0:61438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928a635880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57418:x1875872192905984:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (322us total) trans 0 rc 0/0 00000100:00100000:2.0:1788972474.220885:0:61438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 510 00000100:00080000:3.0:1788972474.221123:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00080000:3.0:1788972474.221149:0:57418:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928b56fb80 x1875872192905984/t0(0) o13->lustre-OST0000-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1788972490 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788972474.221166:0:57418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928b56fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:57418:1875872192905984:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1788972474.730698:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9793c162a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17:57419:1875872192906240:0@lo:400:kworker.0 00000100:00100000:0.0:1788972474.730725:0:57420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928a635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:57420:1875872192906112:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1788972474.735414:0:57418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9793c1628e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:57418:1875872192906496:0@lo:400:kworker.0 00000100:00100000:0.0:1788972474.735554:0:57420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972474.735632:0:57420:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1788972474.735640:0:57420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9793c1628700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:57420:1875872192906624:0@lo:400:kworker.0 00000100:00100000:0.0:1788972474.735670:0:57420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972474.736186:0:57417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9793c1629f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192906368:0@lo:400:kworker.0 00000100:00100000:0.0:1788972474.736219:0:57417:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972474.736294:0:58915:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192906368 00000100:00100000:0.0:1788972474.736302:0:58915:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192906368 00000100:00100000:0.0:1788972474.736351:0:58915:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:0.0:1788972474.736360:0:58915:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192906368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788972474.736407:0:58915:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b56d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192906368:12345-0@lo:400:kworker.0 Request processed in 47us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972474.736414:0:58915:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00080000:0.0:1788972474.736437:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00080000:0.0:1788972474.736478:0:57417:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff9793c1629f80 x1875872192906368/t0(0) o400->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 0 dl 1788972490 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788972474.736491:0:57417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9793c1629f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192906368:0@lo:400:kworker.0 00000100:00100000:0.0:1788972474.736537:0:58918:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192906112 00000100:00100000:0.0:1788972474.736541:0:58918:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192906112 00000100:00100000:0.0:1788972474.736571:0:58918:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 511 00000100:00100000:0.0:1788972474.736577:0:58918:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b56ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57420:x1875872192906112:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1788972474.736601:0:58918:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1788972474.736649:0:58918:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b56ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57420:x1875872192906112:12345-0@lo:13:osp-pre-1-0.0 Request processed in 71us (1108us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972474.736656:0:58918:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 511 00010000:00080000:0.0:1788972474.736681:0:57420:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928a635f80 x1875872192906112/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1788972490 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788972474.736696:0:57420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928a635f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:57420:1875872192906112:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788972474.736725:0:59755:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192906624 00000100:00100000:0.0:1788972474.736730:0:59755:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192906624 00000100:00100000:0.0:1788972474.736761:0:59755:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1021 00000100:00100000:0.0:1788972474.736768:0:59755:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928b56fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:57420:x1875872192906624:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788972474.736803:0:59755:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928b56fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:57420:x1875872192906624:12345-0@lo:400:kworker.0 Request processed in 35us (1131us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972474.736809:0:59755:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1021 00000100:00080000:0.0:1788972474.736822:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788972474.736837:0:57420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9793c1628700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:57420:1875872192906624:0@lo:400:kworker.0 00000100:00100000:1.0:1788972474.737082:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788972474.739516:0:57420:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9793c1629c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:57420:1875872192906752:0@lo:400:kworker.0 00000100:00100000:0.0:1788972474.740266:0:58109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192906240 00000100:00100000:0.0:1788972474.740275:0:58109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192906240 00000100:00100000:0.0:1788972474.740326:0:58109:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 606 00000100:00100000:0.0:1788972474.740336:0:58109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1a8b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17+19:57419:x1875872192906240:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788972474.740409:0:58109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1a8b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17+19:57419:x1875872192906240:12345-0@lo:400:kworker.0 Request processed in 74us (3327us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972474.740429:0:58109:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 606 00000100:00100000:1.0:1788972474.740560:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9793c162a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17:57419:1875872192906240:0@lo:400:kworker.0 00000100:00100000:3.0:1788972474.744147:0:57418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788972474.744571:0:58117:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192906496 00000100:00100000:1.0:1788972474.744581:0:58117:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192906496 00000100:00100000:1.0:1788972474.744647:0:58117:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1022 00000100:00100000:1.0:1788972474.744657:0:58117:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928a635f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:57418:x1875872192906496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788972474.744735:0:58117:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928a635f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:57418:x1875872192906496:12345-0@lo:400:kworker.0 Request processed in 79us (594us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972474.744742:0:58117:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1022 00000100:00100000:3.0:1788972474.744787:0:57418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9793c1628e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:57418:1875872192906496:0@lo:400:kworker.0 00000100:00100000:2.0:1788972474.745375:0:57420:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1788972474.746694:0:58116:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192906752 00000100:00100000:1.0:1788972474.746705:0:58116:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192906752 00000100:00100000:1.0:1788972474.746809:0:58116:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1023 00000100:00100000:1.0:1788972474.746818:0:58116:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1629180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:57420:x1875872192906752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1788972474.746888:0:58116:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1629180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:57420:x1875872192906752:12345-0@lo:400:kworker.0 Request processed in 70us (1513us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972474.746897:0:58116:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1023 00000100:00100000:3.0:1788972474.747021:0:57420:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9793c1629c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:57420:1875872192906752:0@lo:400:kworker.0 00000100:00080000:2.0:1788972474.747174:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1788972475.182127:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1788972475.507184:0:75642:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 12:47:54 (1788972474) 00010000:00010000:1.0:1788972476.202921:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1788972477.038714:0:75690:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type 00010000:00010000:1.0:1788972477.226714:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1788972478.250650:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1788972478.527677:0:57411:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.24@tcp (source: 12345-192.168.203.24@tcp) 00000100:00100000:1.0:1788972478.527729:0:57411:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.24@tcp (source: 12345-192.168.203.24@tcp) 00000100:00100000:1.0:1788972478.527761:0:61016:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872179098112 00000100:00100000:1.0:1788972478.527771:0:61016:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872179098112 00000100:00100000:1.0:1788972478.527827:0:61016:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.24@tcp, seq: 1024 00000100:00100000:1.0:1788972478.527839:0:61016:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1397b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:bb578dba-66a2-4424-b6c3-0338dda03657+6:100242:x1875872179098112:12345-192.168.203.24@tcp:400:kworker.0 00000100:00100000:1.0:1788972478.527905:0:61016:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1397b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:bb578dba-66a2-4424-b6c3-0338dda03657+6:100242:x1875872179098112:12345-192.168.203.24@tcp:400:kworker.0 Request processed in 70us (176us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972478.527929:0:61016:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.24@tcp, seq: 1024 00000100:00080000:1.0:1788972478.527953:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1788972478.528489:0:58107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872179097984 00000100:00100000:3.0:1788972478.528496:0:58107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872179097984 00000100:00100000:3.0:1788972478.528532:0:58107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.24@tcp, seq: 607 00000100:00100000:3.0:1788972478.528539:0:58107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1a89c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:94d249cb-8ce8-4694-9530-465f9f595f02+10:100242:x1875872179097984:12345-192.168.203.24@tcp:400:kworker.0 00000100:00100000:3.0:1788972478.528575:0:58107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1a89c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:94d249cb-8ce8-4694-9530-465f9f595f02+10:100242:x1875872179097984:12345-192.168.203.24@tcp:400:kworker.0 Request processed in 36us (911us total) trans 0 rc 0/0 00000100:00100000:3.0:1788972478.528581:0:58107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.24@tcp, seq: 607 00000001:02000400:0.0:1788972478.533934:0:75738:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-lnet test 350: Check import --exec of ping/discover ========================================================== 12:47:57 (1788972477) 00010000:00010000:3.0:1788972479.277187:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1788972479.344213:0:57417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9793c162aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192906880:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1788972479.344347:0:57417:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788972479.344442:0:58919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192906880 00000100:00100000:2.0:1788972479.344467:0:58919:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192906880 00000100:00100000:2.0:1788972479.344545:0:58919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:2.0:1788972479.344556:0:58919:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c162b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192906880:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1788972479.346512:0:58919:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1788972479.346686:0:58919:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c162b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192906880:12345-0@lo:13:osp-pre-0-0.0 Request processed in 2127us (2336us total) trans 0 rc 0/0 00000100:00100000:2.0:1788972479.346699:0:58919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00010000:00080000:3.0:1788972479.346896:0:57417:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff9793c162aa00 x1875872192906880/t0(0) o13->lustre-OST0000-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1788972495 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788972479.346981:0:57417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9793c162aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192906880:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1788972479.350570:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1788972479.852075:0:57417:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1788972479.852118:0:57417:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928bed7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192907008:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1788972479.852169:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928beace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17:57419:1875872192907136:0@lo:400:kworker.0 00000100:00100000:0.0:1788972479.852264:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972479.852325:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928a49fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:57419:1875872192907648:0@lo:400:kworker.0 00000100:00100000:0.0:1788972479.852357:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972479.852393:0:57419:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1788972479.852399:0:57419:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928beafb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:57419:1875872192907264:0@lo:400:kworker.0 00000100:00100000:0.0:1788972479.852429:0:57419:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1788972479.852510:0:58110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192907136 00000100:00100000:0.0:1788972479.852521:0:58110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192907136 00000100:00100000:0.0:1788972479.852602:0:58110:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 608 00000100:00100000:0.0:1788972479.852613:0:58110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1a8bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17+19:57419:x1875872192907136:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788972479.852675:0:58110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1a8bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17+19:57419:x1875872192907136:12345-0@lo:400:kworker.0 Request processed in 63us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1788972479.852706:0:57418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928a49ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:57418:1875872192907520:0@lo:400:kworker.0 00000100:00100000:2.0:1788972479.852744:0:57418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788972479.852771:0:57418:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1788972479.852778:0:57418:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff97928a49dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:57418:1875872192907392:0@lo:400:kworker.0 00000100:00100000:2.0:1788972479.852803:0:57418:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1788972479.852834:0:58116:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192907648 00000100:00100000:2.0:1788972479.852838:0:58116:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192907648 00000100:00100000:2.0:1788972479.852882:0:58116:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192907520 00000100:00100000:2.0:1788972479.852886:0:58116:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192907520 00000100:00100000:2.0:1788972479.852907:0:58116:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192907392 00000100:00100000:2.0:1788972479.852910:0:58116:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192907392 00000100:00100000:2.0:1788972479.852925:0:58116:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1025 00000100:00100000:2.0:1788972479.852948:0:58116:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928a49c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:57419:x1875872192907648:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788972479.852985:0:58116:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928a49c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:57419:x1875872192907648:12345-0@lo:400:kworker.0 Request processed in 39us (627us total) trans 0 rc 0/0 00000100:00100000:2.0:1788972479.852992:0:58116:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1025 00000100:00100000:2.0:1788972479.853011:0:58116:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1026 00000100:00100000:2.0:1788972479.853016:0:58116:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1629500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:57418:x1875872192907520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788972479.853048:0:58116:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1629500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:57418:x1875872192907520:12345-0@lo:400:kworker.0 Request processed in 32us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1788972479.853055:0:58116:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1026 00000100:00100000:2.0:1788972479.853066:0:58116:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1027 00000100:00100000:2.0:1788972479.853072:0:58116:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff9793c1629880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:57418:x1875872192907392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1788972479.853099:0:58116:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff9793c1629880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:57418:x1875872192907392:12345-0@lo:400:kworker.0 Request processed in 27us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1788972479.853118:0:58116:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1027 00000100:00100000:2.0:1788972479.853174:0:57418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928a49dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:57418:1875872192907392:0@lo:400:kworker.0 00000100:00100000:0.0:1788972479.853188:0:58110:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 608 00000100:00100000:0.0:1788972479.853295:0:58917:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192907264 00000100:00100000:0.0:1788972479.853300:0:58917:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192907264 00000100:00100000:0.0:1788972479.853354:0:58917:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:0.0:1788972479.853363:0:58917:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928545f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:57419:x1875872192907264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1788972479.853473:0:58917:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928545f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:57419:x1875872192907264:12345-0@lo:400:kworker.0 Request processed in 112us (1042us total) trans 0 rc 0/0 00000100:00100000:0.0:1788972479.853545:0:58917:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1788972479.853617:0:57417:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00080000:0.0:1788972479.853671:0:57419:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928beafb80 x1875872192907264/t0(0) o400->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 224/224 e 0 to 0 dl 1788972495 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1788972479.853681:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1788972479.853687:0:58111:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1788972479.853766:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928beafb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:57419:1875872192907264:0@lo:400:kworker.0 00000100:00100000:0.0:1788972479.853780:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928beace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:c2a4ea9e-b180-40d2-b7dd-3b0d82230c17:57419:1875872192907136:0@lo:400:kworker.0 00000100:00100000:0.0:1788972479.853791:0:57419:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928a49fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:57419:1875872192907648:0@lo:400:kworker.0 00000100:00100000:1.0:1788972479.854349:0:61438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1875872192907008 00000100:00100000:1.0:1788972479.854357:0:61438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1875872192907008 00000100:00100000:1.0:1788972479.854406:0:61438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:1.0:1788972479.854416:0:61438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff97928bed5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192907008:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1788972479.854556:0:61438:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1788972479.854631:0:61438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff97928bed5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:57417:x1875872192907008:12345-0@lo:13:osp-pre-1-0.0 Request processed in 217us (1014us total) trans 0 rc 0/0 00000100:00100000:1.0:1788972479.854639:0:61438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00100000:2.0:1788972479.854806:0:57418:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928a49ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:57418:1875872192907520:0@lo:400:kworker.0 00010000:00080000:3.0:1788972479.854949:0:57417:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff97928bed7b80 x1875872192907008/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1788972495 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788972479.854971:0:57417:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff97928bed7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:57417:1875872192907008:0@lo:13:osp-pre-1-0.0 00000001:02000400:0.0:1788972480.070270:0:75787:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-lnet test_350: @@@@@@ FAIL: lnetctl lnet unconfigure failed 240 00010000:00010000:2.0:1788972480.298671:0:58102:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1788972481.323665:0:58103:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 384 lines, 384 kept, 0 dropped, 0 bad.