00010000:00010000:0.0F:1789503492.745241:0:5826:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0F:1789503493.704134:0:3308:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff98fcffe2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3308:1876424853170944:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789503493.704227:0:3308:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0F:1789503493.704903:0:59136:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876424853170944 00000100:00100000:3.0:1789503493.704919:0:59136:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876424853170944 00000100:00100000:3.0:1789503493.704994:0:59136:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2005 00000100:00100000:3.0:1789503493.705004:0:59136:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff98fbc19e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3308:x1876424853170944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789503493.705044:0:59136:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40960000 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789503493.705094:0:59136:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff98fbc19e5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3308:x1876424853170944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (867us total) trans 0 rc 0/0 00000100:00100000:3.0:1789503493.705100:0:59136:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2005 00000100:00100000:3.0:1789503493.705132:0:3307:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff98fcd832b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3307:1876424853171328:0@lo:400:kworker.0 00000100:00100000:2.0:1789503493.705142:0:3308:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff98fce7975f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3308:1876424853171456:0@lo:400:kworker.0 00000100:00100000:3.0:1789503493.705161:0:3307:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789503493.705167:0:3308:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789503493.705191:0:6687:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876424853171328 00000100:00100000:3.0:1789503493.705194:0:6687:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876424853171328 00010000:00080000:2.0:1789503493.705203:0:3308:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff98fcffe2aa00 x1876424853170944/t0(0) o13->lustre-OST0000-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1789503509 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789503493.705222:0:3308:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff98fcffe2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3308:1876424853170944:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0F:1789503493.705302:0:3309:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff98fbc19e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:493c30f9-08db-4337-ae00-4d3725bf750d:3309:1876424853171072:0@lo:400:kworker.0 00000100:00100000:1.0:1789503493.705351:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789503493.705382:0:3309:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff98fce7977100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3309:1876424853171584:0@lo:400:kworker.0 00000100:00100000:1.0:1789503493.705407:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789503493.705427:0:3309:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789503493.705432:0:3309:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff98fcd8328380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3309:1876424853171200:0@lo:400:kworker.0 00000100:00100000:1.0:1789503493.705454:0:3309:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789503493.705480:0:91636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876424853171456 00000100:00100000:1.0:1789503493.705484:0:91636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876424853171456 00000100:00100000:1.0:1789503493.705509:0:91636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876424853171584 00000100:00100000:1.0:1789503493.705512:0:91636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876424853171584 00000100:00100000:2.0:1789503493.705519:0:91629:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876424853171072 00000100:00100000:2.0:1789503493.705522:0:91629:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876424853171072 00000100:00100000:1.0:1789503493.705528:0:91636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876424853171200 00000100:00100000:1.0:1789503493.705530:0:91636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876424853171200 00000100:00100000:2.0:1789503493.705546:0:91629:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1039 00000100:00100000:1.0:1789503493.705546:0:91636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 911 00000100:00100000:2.0:1789503493.705565:0:91629:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff98fbca980050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:493c30f9-08db-4337-ae00-4d3725bf750d+19:3309:x1876424853171072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789503493.705565:0:91636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff98fce7975500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+12:3308:x1876424853171456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789503493.705601:0:91629:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff98fbca980050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:493c30f9-08db-4337-ae00-4d3725bf750d+19:3309:x1876424853171072:12345-0@lo:400:kworker.0 Request processed in 37us (248us total) trans 0 rc 0/0 00000100:00100000:1.0:1789503493.705601:0:91636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff98fce7975500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+12:3308:x1876424853171456:12345-0@lo:400:kworker.0 Request processed in 37us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1789503493.705608:0:91629:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1039 00000100:00100000:1.0:1789503493.705608:0:91636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 911 00000100:00100000:1.0:1789503493.705622:0:91636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 912 00000100:00100000:1.0:1789503493.705628:0:91636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff98fbd0af8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+8:3309:x1876424853171584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789503493.705641:0:91635:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 913 00000100:00100000:2.0:1789503493.705647:0:91635:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff98fbd0af8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+9:3309:x1876424853171200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789503493.705653:0:91636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff98fbd0af8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+8:3309:x1876424853171584:12345-0@lo:400:kworker.0 Request processed in 26us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1789503493.705659:0:91636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 912 00000100:00100000:2.0:1789503493.705670:0:91635:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff98fbd0af8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+9:3309:x1876424853171200:12345-0@lo:400:kworker.0 Request processed in 24us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1789503493.705676:0:91635:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 913 00000100:00100000:1.0:1789503493.705707:0:3309:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff98fcd8328380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3309:1876424853171200:0@lo:400:kworker.0 00000100:00100000:1.0:1789503493.705716:0:3309:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff98fbc19e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:493c30f9-08db-4337-ae00-4d3725bf750d:3309:1876424853171072:0@lo:400:kworker.0 00000100:00100000:1.0:1789503493.705725:0:3309:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff98fce7977100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3309:1876424853171584:0@lo:400:kworker.0 00000100:00100000:1.0:1789503493.705780:0:3308:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff98fce7975f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3308:1876424853171456:0@lo:400:kworker.0 00000100:00100000:3.0:1789503493.705922:0:6687:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2675 00000100:00100000:3.0:1789503493.705930:0:6687:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff98fcc4101f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3307:x1876424853171328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789503493.705978:0:6687:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff98fcc4101f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3307:x1876424853171328:12345-0@lo:400:kworker.0 Request processed in 50us (817us total) trans 0 rc 0/0 00000100:00100000:3.0:1789503493.705984:0:6687:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2675 00000100:00080000:0.0:1789503493.706090:0:5834:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789503493.706095:0:5834:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789503493.706097:0:5834:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789503493.707408:0:3307:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff98fcd832b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3307:1876424853171328:0@lo:400:kworker.0 00010000:00010000:2.0:1789503493.768988:0:5827:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789503493.950684:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:3.0:1789503493.950737:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-192.168.201.51@tcp (source: 12345-192.168.201.51@tcp) 00000100:00100000:1.0:1789503493.950805:0:91636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876424843226496 00000100:00100000:1.0:1789503493.950814:0:91636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876424843226496 00000100:00100000:1.0:1789503493.950884:0:91636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 914 00000100:00100000:1.0:1789503493.950895:0:91636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff98fcc4102300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b01ecd27-8a1a-4bf5-b85f-d66031e73ace+6:2352:x1876424843226496:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:1.0:1789503493.950976:0:91636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff98fcc4102300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b01ecd27-8a1a-4bf5-b85f-d66031e73ace+6:2352:x1876424843226496:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 83us (238us total) trans 0 rc 0/0 00000100:00100000:1.0:1789503493.950984:0:91636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 914 00000100:00100000:2.0:1789503493.951301:0:91651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876424843226368 00000100:00100000:2.0:1789503493.951307:0:91651:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876424843226368 00000100:00100000:2.0:1789503493.951353:0:91651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.51@tcp, seq: 1040 00000100:00100000:2.0:1789503493.951360:0:91651:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff98fbca983850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:c64a0b47-ceaf-4cd5-a1f1-cb7110c8daa7+10:2352:x1876424843226368:12345-192.168.201.51@tcp:400:kworker.0 00000100:00100000:2.0:1789503493.951400:0:91651:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff98fbca983850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0003:c64a0b47-ceaf-4cd5-a1f1-cb7110c8daa7+10:2352:x1876424843226368:12345-192.168.201.51@tcp:400:kworker.0 Request processed in 40us (724us total) trans 0 rc 0/0 00000100:00100000:2.0:1789503493.951604:0:91651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.51@tcp, seq: 1040 00000100:00080000:0.0:1789503493.951721:0:5834:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:02000400:0.0:1789503494.725400:0:114725:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-quota test_16c: @@@@@@ FAIL: mds1: import is not in FULL state after 50 00010000:00010000:3.0:1789503494.791865:0:29402:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789503495.818349:0:20536:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 80 lines, 80 kept, 0 dropped, 0 bad.